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PREFACE 



The last decade has witnessed many concerted efforts to improve the quality of education 
in the health sciences. Some of these efforts have involved groups of schools working 
toward common objectives and developing instructional programs to achieve these 
objectives. Similar activities have also been organized through professional discipline- 
oriented societies such as the American Physiological Society, American Association for 
Cancer Education, American Society of Hematology, and the American Academy of 
Orthopaedic Surgery. These efforts have ranged from the locating and peer reviewing of 
available material to developing comprehensive instruction programs for an entire 
discipline. 

The following publication documents the successes and failures of a comprehensive effort 
by the Steering Committee for Cooperative Teaching. Association of Professors of 
Gynecology and Obstetrics in (1) developing a cooperative teaching network among the 
Gynecology-Obstetrics Departments in United States and Canadian medical colleges, and 

(2) implementing educational techniques to improve instruction. 

This project was built on the premise that for a viable cooperative teaching network to 
succeed, the first need is the development of a diffusion network by which current ideas, 
materials, and information on audiovisual material could be shared. Among the major 
initial tasks were (1) developing a teaching bulletin, (2) peer-reviewing available material, 

(3) disseminating selected audiovisual material, and (4) evaluating audiovisual material 
and teaching strategies. Approximately two years were spent developing the teaching 
network and this effort comprises the first portion of this report. 

The second portion of this report describes a model for the development, evaluation, and 
sharing of new educational materials. The report describes how a consensus for a standard 
curriculum in Obstetrics and Gynecology was obtained and how production of materials 
in these areas is being coordinated among Departments of Obstetrics and Gynecology. 

It is hoped that you find this report of value. Hopefully, other groups can learn from the 
mistakes and successes of this pioneer effort by the Steering Committee for Cooperative 
Teaching Association of Professors of Gynecology and Obstetrics. 

Richard A. Lasco. Ph.D. 
Project Officer 



iii 



I 



ACKNOWLEDGEMENTS 

The Steering Committee for Cooperative Teaching wishes to express its sincere 
thanks to Ms. Deberah Bullias, Viola Higgins, and Carol McBride for typing this report. In 
addition, we deeply appreciate the statistical studies and editing efforts of Mr. Stanley D. 
Pace and Ms. Dolores Thompson. 



iv 



TABLE OF CONTENTS 



CHAPTER PAGE 

1 The Germination of an Idea 1 

2 Early Sharing of Audiovisual Materials 9 

3 Assessment Versus Evaluation: The Politics of Educational Research 

and Educational Innovation 13 

4 The Impact of Innovations in Obstetrics and Gynecology Under- 
graduate Teaching Programs 17 

5 Faculty Reactions to the APGO Teaching Aids Package 25 

6 Student Reactions to the APGO Teaching Aids Package 35 

7 Intraorganizational Cooperation and the Thrust in New Directions . . 43 

8 Conclusions 49 

BIBLIOGRAPHY 53 



v 



LIST OF TABLES 



TABLE TITLE PAGI 

1. Units in Teaching Aids Package 10 

2. Steering Committee for Cooperative Teaching Workshops 15 

3. Changes in Clerkship Characteristics of Undergraduate Teaching 
Programs in Obstetrics and Gynecology, 1970-71 and 1972-73 21 

4. Changes in Evaluation Procedures Used on an Undergraduate 
Clinical Clerkship in Obstetrics and Gynecology, 1970-71 and 
1972-73 22 

5. Changes in Hardware (Machine) Holdings Used in Undergraduate 
Teaching Programs in Obstetrics and Gynecology, 1970-71 and 
1972-73 23 

6. Changes in Software (Teaching Aids) Holdings Used in Under- 
graduate Teaching Programs in Obstetrics and Gynecology, 1970-71 

and 1972-73 24 

7. Faculty Time Spent in Instruction with Undergraduate Medical 
Students in Obstetrics and Gynecology, 1971-72 and 1972-73 26 

8. Comparison by Media Type of Faculty Responses About Familiarity 

with Units in Teaching Aids Package, 1971-72 and 1972-73 27 

9. Comparison by Media Type of Faculty Responses About Avail- 
ability for Student Usage of Units in Teaching Aids Package, 

1971- 72 and 1972-73 28 

10. Comparison by Media Type of Faculty Responses About Requiring 
Students to Use Units in the Teaching Aids Package, 1971-72 and 

1972- 73 29 

1 1 . Comparison by Media Type of Faculty Responses About Units in 
Teaching Aids Package Helping Develop General Knowledge of 
Obstetrics and Gynecology, 1971-72 and 1972-73 29 

12. Comparison by Media Type of Faculty Responses About Units in 
Teaching Aids Package Helping Answer Questions of Residents and 

Staff, 1971-72 and 1972-73 30 

13. Comparison by Media Type of Faculty Responses About Units in 
Teaching Aids Package Helping Answer Questions on a Formal 
Examination, 1971-72 and 1972-73 31 

14. Comparison by Media Type of Faculty Responses About Appro- 
priateness of the Format of Units in the Teaching Aids Package for 

the Conveyance of the Content, 1971-72 and 1972-73 32 

15. Comparison of Faculty Responses to Need for Future Teaching Aids 
Packages and Willingness to Purchase New Materials, 1971-72 and 
1972-73 32 

16. Comparison of Faculty Responses About Overall Rating of the 
Teaching Aids Package, 1971-72 and 1972-73 33 

17. Comparison by Media Type of Student Responses About Using 

Units in Teaching Aids Package, 1971-72 and 1972-73 37 



vi 



LIST OF TABLES (CONTINUED) 



TABLE TITLE PAGE 

18. Comparison by Media Type of Student Responses About Their 
Required Usage of Units in Teaching Aids Package, 1971-72 and 
1972-73 37 

19. Comparison by Media Type of Student Responses About Usefulness 
of Units in Teaching Aids Package, in Developing General 
Knowledge in Obstetrics and Gynecology 1971-72 and 1972-73 38 

20. Comparison by Media Type of Student Responses About Units in 
Teaching Aids Package Helping Answer Questions of Residents and 

Staff, 1971-72 and 1972-73 39 

21. Comparison by Media Type of Student Responses About Units in 
Teaching Aids Package Helping Answer Questions on a Formal 
Examination, 1971-72 and 1972-73 39 

22. Comparison by Media Type of Student Responses About 
Appropriateness of the Format of Units in the Teaching Aids 
Package for the Conveyance of the Content, 1971-72 and 1972-73 .... 40 

23. Comparison of Student Responses About Overall Rating of the 
Teaching Aids Package, 1971-72 and 1972-73 41 

24. Comparison of Student Responses About Need for Future Teaching 

Aids Packages 41 



vii 



CHAPTER 1 
THE GERMINATION OF AN IDEA 



Developments in technology often lead to the development of new institutions 
designed to assist society in gaining the greatest benefits from scientific progress. The 
complexities of the social, economic, political, and educational structures of the North 
American society are at least partially the direct result of new information. As new 
information becomes available, societal institutions, especially those for higher learning, 
tend to create structures for identifying new information and to develop methods for 
disseminating and using it. Such structures, which may assume a variety of forms, subject 
new information to analysis through empirical and experimental research models/67 

One such effort in medical education involving the discipline of obstetrics and 
gynecology concerns a multi-institutional cooperative networking approach. Before 
reviewing activities which led to the development and implementation of this 
multi-institutional cooperative teaching effort, it seems appropriate to review the history 
of medical education and identify technological advances which provided the force for 
such an effort. 

Even before recorded history, man felt the need for some one or more persons in the 
group to care for the sick or injured. It was stated by McGlothlin that the first division of 
labor, in a professional sense, occurred when the societal structure of ancient man 
required two different men to serve the society in two areas of concern which control 
man's fate ( i.e., the spiritual concern and the medical concern). f 27,/ 

European Antecedents 

Medical education, although rooted in the tradition of the Greek physician 
Hippocrates, became formalized near the beginning of the Middle Ages with the 
establishment of the early societies or guilds of scholars and students. The earliest medical 
school was the University of Salerno, Italy, founded during the 11th century. It 
prospered as the center of medical education for several hundred years, taking its place 
beside the University of Paris (Theology) and the University of Bologna (Canon Law). 
(18) This triumvirate of separately located institutions formed the base from which 
Western education emerged/6,/ 

In Europe in the 16th century, most physicians received their education exclusively 
in the universities. Schools of the greatest repute during that period were Italian. Next in 
rank were the French and then the Germans. However, institutions located outside of 
Italy were well attended/^ 

The universities in these various locales were divided into different faculties, 
generally after the nationalities of the faculty and students, leading to so-called 
"nations." Rectors were chosen, one from each nation represented, and a "College of 
Rectors" was formed. The Rectors negotiated with state officials and maintained a power 



which they did not lose until the close of the 16th century. As an example, the German 
nation, especially those from poor classes who were known as traveling scholars, had 
obtained their foundations for university training in Latin schools founded during that 
period in numerous places. Bands of students united together, and during their travels the 
worst barbarities were an everyday occurrence. These traveling scholars became formally 
organized and, like the apprentices of the artisans, were governed by their elder comrades. 
After obtaining rudimentary knowledge necessary for entry into the universities, roving 
students entered into the major institutions in their appropriate nations. Famous teachers 
were received at these institutions with great ceremony. Professors and students stood in 
more friendly relations than probably exist at the present day. Thus, the attachment of 
the students to their professors was secured throughout their life. Many professors were 
physicians of the various princes, who constituted in some cases much of the student 
body of these institutions. They instilled in the students the love of science which 
strengthened and sustained both the professors and the students. They satisfied their 
thirst for science and their efforts after truth and knowledge, often at the neglect of their 
physical needs. These early medical colleges, then, created the forces for promoting the 
sciences and arts and constituted a unique combination of the poor and the rich for such 
advances/^,/ 

Until the close of the 16th century, Italy served as the model for educational 
arrangements. Italy yielded precedence to the Netherlands and to France, whose 
universities at Linden, Paris, and Montpellier then became the goal of all those who 
desired to receive the highest level of training in these departments of science. The 
method of instruction everywhere was the same with the exception of the universities of 
the Netherlands. In most medical schools of the time, the teacher gave simple theoretical 
lectures and dictated appropriate prescriptions. The ordinary language of the colleges was 
still Latin until 1688, when one of the great lecturers of the time ventured to deliver 
lectures in German. There were medical colleges where professors examined patients in 
front of students, made the diagnosis in the presence of students, and determined 
treatments to be provided. However, most medical colleges of the time lectured only on 
theoretical medicine. Student influence on the curriculum during this period dictated 
such policies. However, the seeds of change whereby faculty would set the curriculum 
had their greatest impetus from the Netherlands/^ 

In England, from the middle of the 17th century until 1868, students were required 
to reside in a college or hall; however, clinical instruction was given in hospitals. In 1697, 
an anatomical theater was first erected in "Surgeons Hall" in Edinburgh, Scotland. 
However, occupation with practical anatomy was regarded by many physicians as a 
business unworthy of them, and a separation between the "ordinary physician" who 
practiced surgery and the "higher physicians" who considered surgery beneath their 
dignity existed/ 4) 

Learned societies which had been formed earlier provided great input into the 
medical schools at these and later times, especially as they supplemented the instruction 
at the universities. Many practicing physicians who belonged to these societies lectured at 
the universities through the 17th century. These physicians were not simply savants but 
were very frequently zealous investigators and contributed much to the higher learning of 
the period. The changes which were occurring in medical education during this period had 
great impact on the structure of medical education in the United States and Canada/^ 
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Medical Education in the United States and Canada to 1910 



Although the American college was based upon English models, the establishment of 
the first United States medical college at the College of Philadelphia in l 765 was strongly 
influenced by the University of Edinburgh, Scotland/ 8) The first faculty members at the 
University of Pennsylvania were John Morgan and William Shippen, who had received 
advanced training in medicine at Edinburgh. It should be noted that Benjamin Franklin, 
who contributed so much to the development of American education, was much involved 
in advising Morgan as a young student in the selection of the University of Edinburgh and 
subsequently assisted in raising money for the medical college at Philadelphia/42,/ It was 
over sixty years later that the first Canadian medical college was established at Montreal, 
Quebec, Canada (McGill University Faculty of Medicine, founded in \829).(]4) 

Two years after the establishment of the Medical College at the College of 
Philadelphia, Kings College (later named Columbia University) in New York became the 
second medical college founded in the colonies. Dr. Samuel Clossy, a graduate of Trinity 
College, Dublin, Ireland, had been lecturing in anatomy at Kings College since 1763, and 
he joined with Dr. Samuel Bard, a graduate of Edinburgh, and four other physicians in 
founding the medical college, thereby bringing into early colonial medical education the 
influence of another European institution/42/ Harvard established a medical college in 
1783, and medical colleges were established at Dartmouth in 1798 and at Yale in 1813. 
Early Canadian medical colleges were McGill University Faculty of Medicine, established 
in 1829; University of Toronto Faculty of Medicine, 1843; University of Montreal 
Faculty of Medicine, 1843; Laval University Faculty of Medicine, 1852; and Queens 
University Faculty of Medicine, 1854. These medical colleges represented both French 
and English models of medical education/ 14) These colleges were closely associated with 
their parent colleges — an association which undoubtedly facilitated communication and 
interaction between the medical college and other departments and colleges of the parent 
institution/^ 

Although some medical colleges were established during the early period of American 
higher education, the training of the physician was generally accomplished under an 
apprenticeship system. Brubacher and Rudy reported that: 

The would-be physician started making himself useful by washing bottles, 
later mixing drugs, and, perhaps at a still later stage, progressing to such 
routine matters as blood-letting. By being present in the doctor's office 
and accompanying him on calls, he picked up much of the lore in diagnosis 
and therapy/5,/ 

The period of North American expansion between the years 1800 and 1860 also was 
a period of great expansion of institutions of higher learning//,/ The effect on medical 
education can be observed in the statement made by Abraham Flexner: 

A harmful precedent was established with the foundation early in the 19th 
Century at Baltimore of a proprietary school, the so-called Medical 
Department of the so-called University of Maryland. Before that, a college 
of medicine had been a branch growing out of the living university trunk. 
This organic connection guaranteed certain standards and ideals . . . which 
medical education could, as experience proved, ill afford to forego/ 14) 

Medical education experienced a period of simultaneous expansion and deterioration 
during this period as evidenced by the establishment of about 450 such proprietary 
schools in the United States and Canada. The mode of instruction consisted of a lecture 



3 



hall and a practitioner or two. "it soon became obvious that establishing a proprietary 
medical school was a lucrative enterprise. "(29) As a reaction to the trend, the American 
Medical Association was founded in 1874 by concerned physicians. The organization was 
conceived of as an instrument of reform/42,/ 

The task of the newly created medical association was quite difficult. Charles Eliot, 
president of Harvard from 1869 to 1909, attempted to raise standards at the Harvard 
Medical School in 1870 by proposing to the Medical School that five-minute oral quizzes, 
used as final examinations, be replaced by written examinations. He was told that written 
examinations were impossible for the Medical School as a majority of students could not 
write well enough/75,/ It was reported that the state of medical education at colleges was 
so low that: ". . . at Bowdoin, . . . undergraduates there looked down on medical students 
on its campus with a disdain and were not above playing pranks on them.'Y^ 

During the period 1860 to 1880, several colleges, notably Columbia University and 
the University of Michigan, introduced the experimental method and laboratory facilities 
to the colleges of medicine in an attempt to raise standards/29,/ Similar thrusts 
undoubtedly were occurring in Canada as medical colleges were being established at 
Dalhousie University (1868) and the University of Manitoba (1883)/ 74,1 

The Effects of the Flexner Report: 1910-1950 

The establishment in 1876 of Johns Hopkins University, which was patterned after 
German universities of the times, ushered into the United States' system of higher 
education a rebirth of university-oriented medical education, which had floundered 
shortly after the end of the Colonial Period/^ Flexner was influenced by the medical 
program established at Johns Hopkins University, and his timely but overdue report on 
the state of medical education in 1910 reflected this influence. (14) The Flexner Report 
stimulated in the United States a reorganization of medical education which led to the 
demise of most proprietary schools of medicine and an integration of medical colleges 
with parent institutions by the middle \920\.( 23) 

A new era in medical education was the major result of the Flexner Report. Medical 
colleges either were sponsored by a parent university or quickly associated with a nearby 
university in order to attempt to bring about a medical education program that was more 
comprehensive in its coverage of the basic medical sciences./^,/ In assessing the impact of 
the Flexner Report, Pusey stated: "It was his [Flexner's] conviction that the teaching of 
medicine had to be taken away from proprietary schools, wedded to practice, and 
strongly implanted in universities. "(31) 

Githens reported that the objectives of medical education, following the Flexner 
Report, were to: 

. . . produce physicians having backgrounds in all that was known of 
human biology and competent in all aspects of the art and science of 
medicine . . . This produced the most rigid curriculum imaginable in 
essentially all medical schools for about forty years. It also produced 
faculty attitudes directed toward covering all the materials in all the fields 
of medicine to certify that the graduate would be a reliable physician for 
the publicY 16) 

The trend in medical education from the 1910's to the 1940's was then to stress 
basic medical sciences during the first two years of medical education, clinical medicine 
during the second two years; and provide supervised practice through an internship 
program the year after graduation from medical college. During World War II, 
international problems brought about technological and scientific advancement, which 
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continued at an increased rate after the end of the war. However, the basic structure of 
medical education remained unchanged .(23) 

Forces for the Reorganization of Curriculum: 1950—1970 

The idea that improvement in curriculum was a major need in medical colleges was 
behind Miller's provocative article, "Adventures in Pedagogy.' , (28) A very negative 
rebuttal to such a notion was made by Lymen in "Disaster in Pedagogy. "(25) Forces to 
focus the attention of medical college faculties on the problem of seeking alternative 
means to the "Flexner Model" of medical education became apparent during this period. 

The establishment of centers for research in medical education in a number of 
institutions, initiated in 1956 by Ham at Case Western Reserve and Miller at the 
University of Illinois, focused attention on the needs and characteristics of medical 
students, faculty, and curriculum .(6) A comprehensive study of medical colleges 
attempting radical departures from traditional curriculum patterns was reported by 
Lee. (23) The Association of American Medical Colleges, through a variety of efforts 
closely associated with the thrust noted above, culminated in a formal workshop on the 
medical school curriculum/ 1) 

While the historical perspective of events during this period offers insight into the 
manner in which change became a way of life in medical colleges, it seems best to focus 
on the impact of educational technology as a force leading to the development of a 
"networking approach." In addition, the results of experimentation contrasting 
traditional teaching practices with mediated learning programs seem important in placing 
the idea of networking in an intelligible perspective. 



The Educational Media Movement 

Advocates for improving medical education have viewed technological advances as 
the means for creating more effective and efficient learning programs for medical 
students. Past impacts of individualized medical educational programs using multimedia 
methodologies in both the basic science and clinical science curriculums have been 
reported. 

In 1964, West and Stickley completed an evaluation of the effectiveness of using 
eight millimeter motion pictures supplemented with written programmed instructional 
materials in pharmacology, compared with traditionally operated laboratory learning 
experiences. Scores on a post-test between the two groups of students indicated that 
students who used the self-instructional mediated programs earned higher test scores than 
the control group who completed the traditional laboratory program. However, West and 
Stickley concluded that the cost of producing the eight millimeter motion pictures and 
programmed materials was quite expensive when compared to the cost of materials used 
in the traditional laboratory program .(39) 

Use of a self-instructional program in biochemistry to be used as a precourse 
supplement, reported by Christensen in 1965, showed favorable results in knowledge 
gains by students///,/ It was concluded in another study by Lloyd that students were 
extremely receptive to self-instructional programs involving clinical problem solving, 
particularly programs providing "delayed feedback. "(24) A teaching-machine program 
and a series of lectures in electrocardiography were compared by Andersen, Owen, Hall, 
and Smart, and the results indicated no significant difference in effectiveness or efficiency 
between the two methods. (2) It was concluded by Azneer, Kesler, and Cacamo that a 
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self-teaching program incorporating taped lectures and filmstrips was superior to the 
conventional textbook approach in the learning of diabetic acidosis./^ Mulvihill used 
self-teaching, audiovisual programs as a supplement to the curriculum at Dartmouth 
Medical School and concluded that they were effective for this purpose. He indicated that 
utilization by the students was excellent, and comments received from faculty and 
students were quite favorable. (30) 

A significant study by Feldman in 1969 indicated that medical-student mode of 
performance improved as a result of student utilization of self-instructional programs. ( 13) 
In 1971, McCarthy assessed student perceptions of an individualized self-instructional 
unit in general surgery and concluded that all students involved in the study reported 
"favorable" or "very favorable" comments about these materials./^,/ The effectiveness 
of teaching-machine programs compared to formal lectures in the curriculum of clinical 
medicine was studied by Sweet and Doyle in 1971. This study reported that students 
considered the teaching-machine method superior to a formal lecture. (38) 

During 1973, Guyton completed a multimedia, self-teaching package study involving 
senior medical students at the University of Utah. Twelve self-instructional media units 
were selected for senior-student self-study uses during a preceptorship program in 
practicing physicians 1 offices throughout Utah. Instructional units within the package 
covered topics in physical medicine and rehabilitation, pediatrics, emergency medicine, 
ENT, obstetrics, internal medicine, and human sexual behavior. Of the twelve items 
randomly selected from over 400 media programs identified and obtained, three units 
were from the Association of Professors of Gynecology and Obstetrics Teaching Aids 
Package. An experimental and a control group of senior medical students were selected; 
both groups were administered a pretest, post-test, and delayed retention tests of both 
knowledge and attitudes. Students in the experimental group did not differ significantly 
from students in the control group on the knowledge pretest. However, the experimental 
group performed significantly better (.05 level) on both the post-test and the delayed 
retention knowledge tests. The pretest, post-test, and delayed retention attitudinal tests 
concerning media format indicated little difference between the experimental and the 
control groups, except for their reaction to 16 mm. films with supplemental monograph 
materials. Guyton concluded that students generally had positive attitudes towards the 
programs in the self-teaching package and especially preferred visual media with auditory 
supplementation. He also concluded that programs such as 16 mm. films/supplemental 
monograph and audio cassette/2 x 2 slides were among the most popular. However, these 
programs were considerably more expensive than other programs. He noted that faculty 
must be cognizant of the fact that there are wide differences among student attitudes 
regarding the media formats and that these personal preferences are dependent upon 
multiple criteria/ 1 7) 

These studies indicate that students generally hold favorable attitudes toward the use 
of self-instructional materials. Reports concerning knowledge gains with self-instructional 
materials versus other teaching techniques — lectures, etc. — are conflicting and a case can 
be made for either side of the question. In all probability, the attitudes of faculty toward 
self-instructional materials influence student reactions to such materials in the efforts 
they might expend in using those that are made available to them. While not documented, 
this hypothesis may account for conflicting evidence regarding student knowledge gains 
using various teaching techniques. 

The educational media movement began to take shape in obstetrics and gynecology 
with the publication of Wild's and Zachert's programmed case study approach in 
Lysaught's Programmed Instruction in Medical Education in \964( 40) and of Russell's 
study in the use of audiovisual aids in \966.( 33) In January 1969, Chez reported 
replacement of faculty lectures with "audiovisual booths for the presentation of 
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fundamental information and data, individual tutoring . . "(10) To these reports were 
added Hunter's experience, which indicated extensive use of audiovisual materials in 
learning carrels at the University of Washington. Seattle/ 27 y and Stenchever's concern 
with the need for students to have modality options in their learning programs. (36) These 
stimuli led to the efforts of a small group of dedicated individuals from various 
institutions to work together for cooperative teaching purposes. 

Two major outcomes resulted between 1970 and 1971. Rulin and Chez disseminated 
the "Pittsburgh Package" in early 1970 to fifty-five departments of obstetrics and 
gynecology in medical schools/ and the Steering Committee for Cooperative 
Teaching, Association of Professors of Gynecology and Obstetrics, disseminated a 
Teaching Aids Package to 129 departments of obstetrics and gynecology in the United 
States and Canada the following year. (37) 
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CHAPTER 2 

EARLY SHARING OF AUDIOVISUAL MATERIALS 



For the purpose of discussing cooperative efforts in the use of media in teaching 
programs, an Ad Hoc Committee was founded during July 1969. Representatives of 
departments of obstetrics and gynecology from eight universities met at Case Western 
Reserve University. Four objectives were identified by the Committee. They were: 

1. The sharing of evaluation data concerning the learning effectiveness of teaching 
materials among the members of the Ad Hoc Committee. 

2. The establishment of a Newsletter which would delineate the efforts of the Ad 
Hoc Committee and describe progress in the field of self-instructional materials. 
The Newsletter would be sent to department chairmen of obstetrics and 
gynecology at medical schools throughout the United States and Canada. 

3. The identification of contact persons in all colleges in the United States and 
Canada who would be interested in information and products produced by the 
Committee. 

4. The establishment of cooperative ventures in the identification, development, 
and testing of instructional materials and an agreement to meet in the future to 
review these efforts./7jy 

Construction of the Network for Dissemination of the 
APGO Teaching Aids Package 

The construction of the network went through a number of phases. In the first 
phase, eight professors gathered together at Case Western Reserve University on July 9 
and 10, 1969.* They formed the Ad Hoc Committee for cooperative teaching and wrote 
the objectives listed above. (37) 

Communication among the Ad Hoc Committee members continued through 
telephone conversations and sporadic interchanges as various members gathered in small 
groups at national conferences during 1969 and early 1970. By that time, sufficient 
interest had been generated by previous activities so that a nucleus of individuals from a 
number of institutions throughout the United States and Canada had expressed interest in 
the efforts of the Ad Hoc Committee. 

In January 1970, the Committee's first Newsletter was published and included 
contributions from Committee members concerning the use of learning materials in their 
departmental teaching programs. The Committee then met in February 1970 during the 
annual conference of the Association of Professors of Gynecology and Obstetrics 
(APGO). It was at this time that the Steering Committee for Cooperative Teaching. 



♦Funds for this meeting were partially provided for by the Carnegie Corporation of New York. 
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Association of Professors of Gynecology and Obstetrics, was established and additional 
members were added. 

Newsletters were published throughout 1970 and early 1971 and included Steering 
Committee membership review of audiovisual materials and devices. A working 
philosophy had been established by the Steering Committee to utilize those modalities 
which did not require extensive investment in hardware. The determination was made to 
enable the Steering Committee to put together a sampler package of audiovisual materials 
which could be used by all institutions in the United States and Canada. 

Throughout 1970, Committee members continued to refine materials they were 
developing, continued to identify excellent materials produced by both commercial and 
noncommercial producers in obstetrics and gynecology, and approached and obtained in 
February 1971 support from the Lister Hill National Center for Biomedical 
Communications, National Library of Medicine, to fund a Cooperative Teaching Project 
Center located at the University of Utalh Concurrent with these activities, additional 
institutions expressed interest in the activities of the Steering Committee. 

During the period February 1970 to February 1971, largely through the efforts of 
the individual Steering Committee members, personal contacts with other faculty 
members, and its Newsletter, 66 institutions expressed an interest in participating in the 
Committee activities. During March of 1971, nonparticipating departments were invited 
by means of a postcard survey to join the network. By June 1971, 116 medical college 
departments of obstetrics and gynecology were included in the network. Since that date, 
14 additional colleges have been added to bring the total number of colleges participating 
in the Cooperative Teaching Network to 130. During that period, the title of the 
Newsletter was changed to the Cooperative Teaching Bulletin, published by the Steering 
Committee for Cooperative Teaching, Association of Professors of Gynecology and 
Obstetrics. Three previous Newsletters had been sent out. However, with the 
establishment of the Cooperative Teaching Project Center, this communication device was 
put on a quarterly basis and sent to all medical school deans, department chairmen of 
obstetrics and gynecology, and faculty contact persons at institutions in the network. 

The Teaching Aids Package selected at the February 1971 meeting of the Steering 
Committee consisted of 22 teaching aids in obstetrics and gynecology. The Steering 
Committee felt the aids were a sample of material relevant for undergraduate medical 
students and were technically excellent with regard to mode of presentation, ease of use 
by students, and quality of production. Included in the Teaching Aids Package were five 
workbooks, one syllabus, three audiotapes, six audiotapes with slides, two programmed 
case studies, and five motion pictures. Table 1 is a listing of items included in the 
Teaching Aids Package. 



TABLE 1 . UNITS IN TEACHING AIDS PACKAGE 



Type Aid 



Title, Author(s) and Institution Where Developed 



Programmed Textbook 



Application of Gynecologic Oncology by Wilds, P. L., 
and Zachert, V., Medical College of Georgia 



Programmed Textbook 



Essentials of Gynecologic Oncology by Wilds, P. L., 
and Zachert, V., Medical College of Georgia 



Workbook 



Labor: A Workbook in Obstetrics and Gynecology by 
Stenchever, M. A., and Kitay, D. Z., Case Western 
Reserve University 
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Workbook 

Programmed Workbook 

Programmed Case 
Programmed Case 
Syllabus 

Tape-Slide Program 

Tape-Slide Program 

Tape-Slide Program 
Tape-Slide Program 

Tape-Slide Program 
Tape-Slide Program 
Audiotape Program 

Audiotape Program 

Audiotape Program 

Motion Picture Film 
Motion Picture Film 

Motion Picture Film 
Motion Picture Film 

Motion Picture Film 



Human Sexual Behavior: A Workbook in 
Reproductive Biology by Stenchever, M. A. Testing 
by Stickley, W. T.. Case Western Reserve University 

Workbook in Obstetrics and Gynecology by Moore, 
J.G., University of California at Los Angeles, as 
modified by Wilds. P. L., Medical College of Georgia 

Abdominal Enlargement With Ophelia Zop by Kelly, 
J. V.. University of Pennsylvania 

Premature Delivery With Mrs. Esmeralda Burp by 
Kelly, J. V., University of Pennsylvania 

Syllabus, for Use With the Carter Model by Carter, J.. 
Indiana University 

Human Sexual Response by Cbez. R. A., University 
of Pittsburgh 

Maternal Physiology by Chez. R. A., University of 
Pittsburgh 

Infertility by Chez, R. A., University of Pittsburgh 

Endocrinology of Pregnancy by Chez, R. A.. 
University of Pittsburgh 

Contraception by Chez. R. A., University of Pittsburgh 

Pelvic Mass by Johnson, W. L., Indiana University 

Antenatal Care With Agent 38-24-34 by Kelly, J. V., 
University of Pennsylvania 

Difficult Labor With Mrs. Jer C. Bounce by Kelly, 
J. V., University of Pennsylvania 

Postpartum Hemorrhage With Gina Lollipop by 

Kelly, J. V., University of Pennsylvania 

Postpartum Hemorrhage by Ortho Pharmaceuticals 

The Female Pelvic Examination by Hunter. C. A., Jr., 
Indiana University 

Normal Delivery by Hunter, C. A., Jr.. Indiana 
University 

Normal Labor by Hunter. C. A., Jr.. Indiana 
University 

Prolongation of Labor Due to Uterine Dysfunction 
by Hunter, C. A., Jr.. Indiana University 



These materials were seen as samples of various modalities which could be used for 
instructional purposes. In addition, materials had previously been approved by content 
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exports on the Steering Committee as effective, efficient learning materials. All items had 
been shared among the members of the Steering Committee and tested on their students, 
and Committee members found these materials useful in their departmental teaching 

programs. 

Decentralized Centralization 

During May 1971, the Steering Committee met and agreed to a districting plan of the 
United States and Canada based on geographical lines. Each member of the Committee 
became a locally available consultant to medical college departments in his area. The 
National Medical Audiovisual Center agreed to accept copies of materials from producers, 
package the materials, and mail materials to network participants. In addition to the 
Cooperative Teaching Bulletin, the link which formed the strongest bond for the network 
was the dissemination of the Teaching Aids Package. During July and August of 1971, 
materials were mailed to 1 16 medical school contact persons. During the period August 
1971 to May 1973, additional copies of the Teaching Aids Package were sent to 13 new 
United States medical school departments of obstetrics and gynecology and to nine 
institutions located outside of the United States boundary. Five of these were sent to a 
medical college in Canada, which placed the Teaching Aids Package in all of its teaching 
hospitals; one was sent to the Pan American Conference for Medical Education in Bogota, 
Colombia; three packages were sent overseas to medical schools in other nations for 
demonstration and teaching purposes. One United States medical school obtained two 
additional packages. In all, 139 Teaching Aids Packages were disseminated. Two sample 
packages were displayed, one at the Lister Hill National Center for Biomedical 
Communications and one at the National Medical Audiovisual Center. These centers are 
located in different geographical areas, but both are parts of the National Library of 
Medicine. 

The dedicated efforts of the small group of persons who initiated the Ad Hoc 
Committee to share instructional materials grew to capture the attention of medical 
educators in obstetrics and gynecology in all medical schools in the United States and 
Canada and four medical education units outside of the North American continent. This 
is evidenced by the number of institutions who presently have the Teaching Aids Package 
and by the number of persons who have asked to be placed on the mailing list to receive 
the Cooperative Teaching Bulletin which now goes to over 50Q persons. The results of the 
dissemination provide insight into the feasibility of such an effort and its impact on 
undergraduate obstetrics and gynecology teaching and learning. 
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CHAPTER 3 

ASSESSMENT VERSUS EVALUATION: THE POLITICS OF 
EDUCATIONAL RESEARCH AND EDUCATIONAL INNOVATION 

A distinction between assessment and evaluation needs to be made in order to 
identify the conceptual framework which formed the initial base for the approach taken 
by the Steering Committee for Cooperative Teaching in assessing the impact of the 
Teaching Aids Package on medical college departments of obstetrics and gynecology in 
the United States and Canada. 



Evaluation 

There exists in medical education a strong desire for evaluation .(!())( 7) New 
approaches have been proposed which would measure the outcomes of education in terms 
of changes in student performance on a scientifically based prescription. These 
approaches would attempt to guarantee the product produced by an educational system. 
That is, professors would have to face up to the demand for accountability to guarantee 
student learning as a prime goal. (5) The need for closely controlled experimental 
approaches is important. 

Assessment 

A necessary first step in assessment is the collection of empirical data and its analysis. 
Assessment has been defined as "the determination over time of the outcomes of 
education in light of different contingencies.' YZ?y Assessment, then, serves two purposes: 
(a) the politics of using information for decision-making (accountability) and (b) research 
(empirical). Some have charged that "assessment is really politics masquerading as 
research. "(19) Educational programs, then, usually serve the interests of competing 
groups. 

A more pristine second component of the assessment process can be found in the 
word "research," which develops the evidence concerning the efficacy of any approach. 
Confusion exists when decision makers equate monitoring of a project with evaluation of 
a project. As stated by House, "auditors end up trying to evaluate and evaluators end up 
attempting to audit programs.'Y/Py Such concepts lead to a determination of the 
rationale for any assessment program. Is it evaluation for evaluation's sake, or evaluation 
for control, or evaluation for aiding decision-making? The selection of the question sets 
the tone and assessment approach for a program. 
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Assessment Approach 



Due to the diversity of the distribution of the Teaching Aids Package, the Steering 
Committee recognized the futility of attempting to develop a strictly experimental 
evaluation design. As virtually all departments of obstetrics and gynecology in medical 
schools in the United States and Canada were a part of the project, it was impossible to 
establish adequate control groups for comparative purposes. When planning for survey 
procedures, the Steering Committee decided that the assessment program should attempt 
to determine the impact of the teaching materials distributed by the Steering Committee 
on departmental teaching programs. Parameters for the assessment program were 
developedYJ7,/ Means were developed to monitor teaching programs existing in 
departments prior to the distribution of the Teaching Aids Package and to later reappraise 
teaching programs and compare these data to note changes. The attitudes of faculty 
about the materials in the Teaching Aids Package were assessed over a two-year period to 
determine if materials would remain viable. A group of students from a number of 
institutions in each district was surveyed over a two-year period to insure student 
opportunity to react to materials. 

In view of the above, The Steering Committee attempted to gather data, analyze it, 
and use it to aid decision making concerning directions and approaches appropriate to 
meeting the needs of network participants. It was recognized that in order for the 
network to remain viable, Steering Committee efforts had to be sensitive and responsive 
to actual problems faced by a large group of faculty from many institutions. 



Assessment Techniques 

During the 1971 annual meeting of the Association of Professors of Gynecology and 
Obstetrics, the Steering Committee met and the basic formulation of the assessment 
design as outlined above was determined. During the following seven months, three 
instruments were developed. These were: (a) the "Faculty Assessment of the Teaching 
Aids Package" form, (b) the "Student Assessment of the Teaching Aids Package" form, 
and (c) the "Teaching Program Assessment" form. 



Evaluation, Teaching and Evaluation Workshop 

Following the development of instruments to assess program parameters and 
faculty/student reactions to the Teaching Aids Package, a number of techniques to gather 
uata were used — mailings, telephone survey techniques, and district workshops. The 
most effective means to gather data proved to be district evaluation workshops. The 
location, dates, and number of participants who attended district workshops are shown in 
Table 2. 

At the workshops, emphasis was placed on obtaining survey data from the faculty 
representatives attending. In addition, the groups focused on the sharing of ideas and 
techniques for improving instruction. Educational consultants were secured to add 
professional educational input to participant discussions. 

A recent consequence of the workshops has been the development and testing 
through cooperative efforts of new instructional materials produced by workshop 
participants. A number of products are being produced and tested at the present time for 
possible use throughout the United States and Canada via the existing network 
capabilities. 
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TABLE 2 STEERING COMMITTEE FOR COOPERATIVE 
TEACHING WORKSHOPS 



Location 


L/alCa 


M 1 1 \w \\or A f 1 n H i n n 
l^UIUUCI \llcllUIHU 


University of California at 
Los Angeles 


Hpppmhpr 10 1Q71 


Q 


Stanford University 


June 14-17, 1972 


12 


University of Minnesota 


November 21-22, 1972 


15 


University of Indiana 


November 30, 1972 


13 


Lister Hill National Center for 
Biomedical Communication 
National Library of Medicine 


December 1. 1972 


14 


University of Utah 


December 8-9, 1972 


9 


University of Pennsylvania 


December 1 1, 1972 


15 


University of Pittsburgh 


January 26. 1973 


15 


University of Chicago 


May 7, 1973 


10 


Michigan State University 


June 4-6, 1973 


13 


University of Oklahoma 


June 27, 1973 


18 




Total Participants 


143 



Data concerning faculty reactions were collected for the academic years 1971-1972 
and 1972-1973. These data will be presented later. However, faculty reactions to the 
Teaching Aids Package in 1971 with specific data were reported by the Steering 
Committee. (35) Major conclusions reached from that study were: 

1 . A majority of faculty members responding to the questionnaire were 
senior members at participating institutions. This indicated a high 
level of faculty concern with new teaching methods in general and 
with the Teaching Aids Package in particular. 

2. Most respondents felt that materials in the Teaching Aids Package 
were helpful to students in acquiring necessary knowledge about 
obstetrics and gynecology. 

3. Students were being given an opportunity to use the materials; 
however, they were not required to do so. Students had available to 
them some alternatives in educational efforts. 

4. Materials appeared to be useful in fulfilling teaching objectives, as 
little negative criticism was presented and a large majority of 
respondents indicated that more teaching material should be produced 
and made available. A majority of the faculty state that they would 
spend departmental funds to purchase additional materials, and, 
indeed, some departments had purchased additional items from the 
original Teaching Aids Package to expand their use in teaching 
programs. 
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5. Many faculty made suggestions as to content areas for which materials 
were not included in the original Teaching Aids Package and for which 
a need existed. 

6. Modality preferences of faculty members were noted by the Steering 
Committee. This confirmed an original bias of the Steering Committee 
that different faculty members would prefer different modalities 
depending upon their local circumstances/ 

To date, information about undergraduate teaching programs in departments of 
obstetrics and gynecology, faculty changes over time, and student reactions to the 
Teaching Aids Package have not been reported. 



16 



CHAPTER 4 

THE IMPACT OF INNOVATIONS IN OBSTETRICS AND GYNECOLOGY 

UNDERGRADUATE TEACHING PROGRAMS 

One means of determining the impact of an educational innovation is to focus on 
output results caused by the stimulus provided. The Steering Committee was cognizant of 
the difficulty of attempting to associate causally educational changes with the Teaching 
Aids Package dissemination over a three-year period (1970-7 1 to 1972-73). It was evident 
that educational incentives from efforts by other groups as well as spontaneously created 
programs, could evoke educational improvement. Results have been reported by Rulin 
and Chez from the "Pittsburgh Package," distributed to 55 medical schools prior to the 
dissemination of the Teaching Aids Package. (32) A five-year study of teaching programs 
by Johnson also may have influenced changes. (See Appendix A.) In addition, during the 
dissemination and implementation period, the American Association of Obstetricians and 
Gynecologists Foundation, Incorporated (AAOG) through the Kennedy Foundation 
Curriculum Committee, published a Guide to a Basic Curriculum in Obstetrics and 
Gynecology. This publication listed 40 terminal objectives and a number of enabling 
objectives. It included case studies representing the basic concepts determined by thai 
Committee as the basis for an undergraduate obstetrical and gynecological teaching 
program. (20) 

These efforts, fully embraced by the Steering Committee membership, certainly 
influenced the educational changes which were noted by the Steering Committee over the 
study period. In addition, a distribution of instructional materials by the Council on 
Residency Education in Obstetrics and Gynecology (CREOG), the publication of the 
motion picture catalog by the American College of Obstetrics and Gynecology (ACOG), 
and the efforts of the Southern Medical Schools Consortium in developing and 
disseminating self-instructional materials, probably contributed to the changes noted. 

Purpose and Methodology 

It was the purpose of the teaching programs assessment study to monitor the impact 
of the Teaching Aids Package on teaching programs in participating departments of 
obstetrics and gynecology. As noted, other groups undoubtedly influenced some of the 
changes found. Unfortunately, there is no realistic manner in which the impact can be 
measured accurately and credit appropriately given to each group involved based upon 
each group's contribution. Even with such problems, it was determined that the 
information was worth gathering and reporting and that the efforts of the Steering 
Committee in its dissemination contributed in some way to changes found. 

Data were collected for the academic year 1970-71 prior to the period when the 
Teaching Aids Package was distributed. A second assessment of teaching programs was 
made during the 1972-73 academic year. Therefore, the two sets of data represented a 
three-year study. 



17 



Instruments Used and Returns Obtained 

During the period June 1971 to October 1971, the Steering Committee devised a 
methodology for collecting information regarding teaching programs in the participating 
institutions. This assessment occurred concurrently with the dissemination of the 
Teaching Aids Package. Contact persons were asked to respond for the year previous to 
its arrival. The Steering Committee was interested in four descriptive areas concerning 
teaching programs in obstetrics and gynecology. The first area determined characteristics 
of the clerkship with respect to students and student/faculty contact hours; the second 
was a description of the evaluation instruments used in each department; the third was a 
list of the hardware holdings within the department for educational purposes; and the 
fourth was a list of the software or other educational media held and used by the 
departments. During the second year of the project, this instrument was revised 
somewhat but still sought the same information. 



Assessment Results 

During the 1970-71 academic years, 97 departments reported about their teaching 
programs. At that time, there were 116 medical schools in the Cooperative Teaching 
Network so the return rate was 84 percent. During the 1972-73 academic year, 95 
departments reported on their teaching programs. Although the network had grown to 
130 schools, 14 of the schools were new to the network and did not have students, 
thereby reducing the number of teaching program assessment possibilities to 116, which 
represented an 82 percent return rate. 



Statistical Treatment of Data 

As noted earlier in Chapter 3, the nature of the teaching program study as well as 
information collected from faculty and students (see Chapters 5 and 6) was (a) to gather 
baseline descriptive information about undergraduate learning programs in obstetrics and 
gynecology through empirical research with an emphasis on assessing faculty and student 
reactions to the Teaching Aids Package, and (b) to assist the Steering Committee in 
making decisions about activities they should undertake to enhance educational 
innovations and to further the use of the newer educational technology for departments 
in the network. During the study period, it was not feasible or appropriate, given the 
"state of the art" in departments of obstetrics and gynecology in all medical colleges in 
the United States and Canada, to attempt an experimental-control group research 
program. The approach taken by the Steering Committee was to demonstrate to faculty, 
and hopefully to some students, a variety of mediated learning strategies. Each unit in the 
Teaching Aids Package was independent of all other units. A post-test for all units was not 
available and, in all probability, would not be valid if one were available since a "Sampler 
Package" is very different from an integrated instructional package based on specific 
objectives. 

All data reported are descriptive in nature. In addition, as assessments occurred over 
time, the purpose was empirical descriptive research. While percentage figures and, in 
some cases, means were determined, assessment of change over time under varying 
conditions could best be determined using the Chi-square statistic. This statistical 
technique allows data to be cast in a matrix where each cell can be analyzed to determine 
the direction of changes noted, especially when statistical significance using the 
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Chi-square is found. (41) 

Both the .05 and the .01 levels of significance were considered as criteria for data 
analyzed. The .05 level means there exists a five percent possibility that changes observed 
are due to chance or uncontrolled influences. Therefore, changes reaching the .05 level 
had a statistical confidence level of 95 percent, and the changes reaching the .01 level had 
a statistical confidence level of 99 percent/47 J 

It should be noted that analyzed data which did not reach the .05 or .01 levels using 
the Chi-square statistic could be predictive of trends or changes occurring. However, a 
minimum level was set at the .05 level as it is generally used as the criterion point in 
educational research/47,) 

Clerkship Characteristics 

Over a three-year period, several changes were noted in clerkship characteristics. 
These changes are shown in Table 3. 

A Chi-square statistic was used to determine differences between data for academic 
years 1970-71 and 1972-73. A significant Chi-square for scheduled student-faculty 
contact hours per week was found. The direction indicated that faculty were spending 
significantly more time with students during the second assessment period. 

Changes in Evaluation Procedures 

Changes were also found in evaluation procedures used by departments. Data are 
shown in Table 4. 

A significant Chi-square was found for departments using written essays as a 
measuring instrument. The direction for this difference was that departments, as a whole, 
were using fewer written essay questions in their evaluation program. Another significant 
Chi-square was found for departments using rating sheets for clinical performance. The 
direction for this change indicated that fewer departments were using rating sheets for 
clinical performance as part of their evaluation program. 

Changes in Hardware (Machine) Holdings 

Changes in departmental holdings of hardware (machines) are shown in Table 5. 

Although significance at the .05 level or beyond as measured by Chi-square statistics 
was not found in hardware holdings, several areas of growth and one area of decrease 
were noted. Departments increased ownership of slide projectors and learning carrels, and 
decreased use of learning carrels in the medical school library. 

Changes in Software (Teaching Aids) Holdings 

Changes in software holdings, e.g., motion pictures, tape and tape-slide programs, 
etc., were found. Data are shown in Table 6. 

A significant Chi-square was found for nontextbook or nonjournal software 
ownership. The direction indicated that more departments had these types of materials 
available than in the first assessment period. A second significant Chi-square was found 
for models owned by departments. The direction indicated that more departments owned 
teaching models during the second assessment period. It should be noted that software 
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sent in the Teaching Aids Package was not counted for the 1972-73 academic year and 
that these data indicate that departments have significantly acquired new and other 
materials. 



Summary 

Effective changes were brought about by the dissemination of the Teaching Aids 
Package modified, as indicated earlier, by other forces interacting on the discipline of 
obstetrics and gynecology. These changes were found to be significant at or beyond the 
.05 level as measured by the Chi-square statistic and are listed below: 

1 . There was an increase in student-faculty contact hours. 

2. There was a decrease in the number of departments using written essay questions 
for evaluation purposes. 

3. There was a decrease in the number of departments using rating sheets for clinical 
performance in departmental evaluation programs. 

4. There was an increase in nontextbook or nonjournal software acquisitions in 
departments even after excluding materials sent to departments in the Teaching Aids 
Package. 
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TABLE 3 CHANGES IN CLERKSHIP CHARACTERISTICS OF 
UNDERGRADUATE TEACHING PROGRAMS IN 
OBSTETRICS AND GYNECOLOGY, 1970-71 
(N = 97 ) AND 1 972-73 (N = 95 ) 





Mean 


Percent Yes 


Percent No 


Category 


(70-71) (72-73) 


(70-71) (72-73) 


(70-71) (72-73) 


Number of Students in 
Clerkship 


17 17 






Number of Clerkships Held Per 
Year 


* ~j 






Clerkship Is Required (Yes) or 
Elective (No) 




* 9? 


* 8 


Length of Clerkship (In Weeks) 


6 7 






**Scheduled Student-Faculty 
Contact, Hours Per Week 


15 35 






Operating Room Time 


* 7 






Clinic Time 


* 11 






Seminar Time 


* 4 






Lecture Time 


* 3 






Rounds Time 


* 4 






Tutorial or Preceptor 
Time 


* 4 






Time Spent Other 
Activities 


* 2 






Departmental Objectives in 
Written Form 




70 61 


30 39 



*Data not available for 1970-71 
**Significant beyond the .01 level 
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TABLE 4 CHANGES IN EVALUATION PROCEDURES USED ON AN 
UNDERGRADUATE CLINICAL CLERKSHIP IN 
OBSTETRICS AND GYNECOLOGY, 
1970-71 (N = 97) AND 1972-73 (N = 95) 





Percent Yes 


Percent No 


Category 


(70-71) 


(72-73) 


(70-71) 


(72-73) 


Departments Using Objectives Tests 


39 


52 


61 


48 


*Departments Using Written Essays 


56 


25 


44 


75 


Departments Using Oral Examinations 

or Fapnltv Fvaliiatinns 


77 


77 


23 


23 


*Departments Using Rating Sheets for 
Clinical Performance 


93 


64 


7 


36 


Departments Using Letter Grades 


27 


14 


73 


86 


Departments Using Number Grades 


21 


16 


79 


84 


Departments Using Pass/Fail/Honors 
Grading System 


77 


79 


23 


21 



*Significant beyond the .01 level 
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TABLE 5 CHANGES IN HARDWARE (MACHINE) HOLDINGS USED 
IN UNDERGRADUATE TEACHING PROGRAMS IN 
OBSTETRICS AND GYNECOLOGY, 1970-71 
(N = 97) AND 1972-73 (N = 95) 





Percent Yes 


Percent No 


Category 


(70-71) 


(72-73) 


(70-71) 


(72-73) 


Departments Owning Motion Picture 
Projector(s) 


77 


80 
o\j 




">0 


Departments Owning Slide 
Projector(s) 


86 


94 


1 4 


u 


Departments Owning Audiotape 
Player(s) 


11 


Q 1 
O J 


19. 
Zo 


i 7 
1 / 


Departments Owning Videotape Player(s) 
or Monitor(s) 


* 


38 


* 


62 


Departments Owning Computer 
Terminals 


* 


10 


* 


90 


Departments Owning Overhead 
Projector(s) 


* 


33 


* 


67 


Departments Owning Learning 
Carrel(s) 


59 


68 


41 


32 


Departments Using Learning Carrel(s) 
in Library, Etc. 


31 


18 


69 


82 



*Data not available for 1970-71 
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TABLE 6 CHANGES IN SOFTWARE (TEACHING AIDS) HOLDINGS 
USED IN UNDERGRADUATE TEACHING PROGRAMS 
IN OBSTETRICS AND GYNECOLOGY, 1970-71 (N = 97) 
AND 1972-73 (N = 95) (Software Sent in APGO Teaching 
Aids Package Not Included) 





Mean 


Percent Yes 


Percent No 


Category 


(70-71) (72-73) 


(70-71) (72-73) 


(70-71) (72-73) 


*Non-Textboofc or Journal 
Software in Departments 


3 


26 


68 


98 


32 


2 


Motion Picture Films Owned 
by Departments 


1 


2 


57 


56 


43 


44 


Tape-Slide Programs Owned 

uy L/t uai iiiidiio 


15 


17 


64 


70 


36 


30 


Programmed Texts Owned 

uy L/Cpdl lllicillo 


i 
i 


i 
i 




35 

o j 


/u 


OD 


Audiotape Programs Owned 
by Departments 


1 


3 


33 


38 


67 


62 


Videotape Programs Owned 
by Departments 


1 


1 


23 


24 


77 


76 


*Models Owned by Department 


1 


2 


40 


76 


60 


24 


Handouts Given to Students 
by Departments 


** 


2 


** 


49 


** 


51 


Other Media Software Owned 
by Departments 


** 


3 


** 


100 


** 


0 



♦Significant at .01 level 
**Data not available 
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CHAPTER 5 

FACULTY REACTIONS TO THE APGO TEACHING AIDS PACKAGE 

The Steering Committee wished to determine the reactions of faculty members in 
departments of obstetrics and gynecology in medical schools in the United States and 
Canada to the instructional units in the Teaching Aids Package. Earlier, in Chapter 3, 
conclusions reached during the first year of the dissemination regarding faculty reactions 
were reported. (See also reference 25.) 

Purpose, Methodology, and Return 

Regardless of the viability of any unit of prepackaged instructional materials, the 
reactions of the faculty to such materials will depend upon its acceptance and use. The 
Steering Committee had designed the original Teaching Aids Package to provide for 
faculty examples of various modalities which could be used in teaching programs in order 
to indicate to them multiple options available for their use in enhancing teaching 
programs. Since the goal was to demonstrate to faculty a variety of modalities requiring 
little hardware gear-up, no effort was made to integrate each Teaching Aids Package unit 
in terms of meeting an overall instructional goal. Rather, each unit selected for inclusion 
in the package was independent of other units within the package in terms of 
instructional objectives. Therefore, the Teaching Aids Package did not constitute a 
unified core of information which could be used as basic didactic information in 
obstetrics and gynecology. 

The purposes of the assessments conducted during academic years 1971-72 and 
1972-73 were to obtain a sample of faculty reactions to the Teaching Aids Package, to 
determine if faculty showed specific modality preferences, and to see if they found 
materials included in the Teaching Aids Package useful in enhancing undergraduate 
instructional programs. In addition, the assessments sought other content areas for which 
materials needed to be developed, inquired whether faculty were willing to purchase new 
materials out of departmental funds if future packages were to be developed, and 
requested an overall evaluation of the Teaching Aids Package. 

During academic year 1971-72, Steering Committee members and consultants from a 
number of Research in Medical Education Centers worked to develop a survey form 
which would yield information concerning the questions noted above. After several drafts 
and revisions, the "Faculty Assessment of the Teaching Aids Package" form was accepted 
by the Steering Committee. These forms were administered to contact persons from 
departments of obstetrics and gynecology who had received the Teaching Aids Package. It 
was felt that the person who received the package would be in the best position to 
evaluate the units contained within it. In some cases, faculty were sent forms by mail and 
were asked to complete and return them to the Project Office. In other cases, faculty 
completed forms during a series of evaluation workshops held in their geographical 
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regions. Follow-up mailings were made in order to obtain as large a response pattern as 

possible. 

The first assessment period, 1971-72, yielded returns from 87 of the 116 
departments which had received the Teaching Aids Package, for a return rate of 75 
percent. As previously indicated, additional packages were mailed during the 1972-73 
academic year. In all, 129 packages were sent to departments of obstetrics and 
gynecology in medical schools in the United States and Canada. Of the 129 schools, 124 
schools were asked to respond to the Faculty Assessment of the Teaching Aids Package 
form. The five remaining schools were not asked to respond because the analysis of 
collected information was in progress when materials were sent. Of the 124 forms 
distributed, 103 were returned for a return rate of 89 percent for the 1972-73 academic 
year. 



Changes in Student-Faculty Contact Hours 

The significant increase in scheduled student-faculty contact hours per week was 
discussed earlier in the review of teaching program assessment studies. Amplification of 
this point is noted in Table 7. 

It should be noted that only one faculty member from each department was asked to 
complete the Faculty Assessment of the Teaching Aids Package form. Generally, persons 
who attended workshops completed these forms, and their responses did not reflect how 
each faculty member in his department could have responded. The information, then, is a 
sample of those faculty who seemed to be the most involved in the undergraduate 
teaching program at their institutions. 



TABLE 7 FACULTY TIME SPENT IN INSTRUCTION WITH 
UNDERGRADUATE MEDICAL STUDENTS IN 
OBSTETRICS AND GYNECOLOGY, 1971-72 AND 
1972-73 



Category 


Percent Faculty 
1971-72 (N = 87) 


Percent Faculty 
(1972-73 (N= 103) 


Percent 
Difference 


Zero Percent 


0 


1 


+ 1 


1 to 25 Percent 


31 


38 


+7 


26 to 50 Percent 


35 


28 


-7 


51 to 75 Percent 


9 


10 


+ 1 


76 to 100 Percent 


0 


6 


+6 


No Response 


25 


17 


-8 


Totals 


100 


100 
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Chi-Square Comparison 



A Chi-square statistic program was used to determine if significant differences in the 
data reported in 1971-72 and 1972-73 existed. There were areas where significant 
statistical differences were noted. The Chi-square studies were limited to media categories 
in order to determine if modality preferences were indicated. However, changes in faculty 
reactions to each unit in the Teaching Aids Package are shown in Appendix B. 

Percentages were computed by multiplying the number of units in a media category 
by the number of usable forms returned, then dividing that value into the number of 
respondees who answered the item "yes," and then multiplying by 100 to obtain a 
percent value. 



Faculty Familiarity with Teaching Aids 

Chan ges in faculty familiarity with units in the Teaching Aids Package were found. 
The changes are shown in Table 8. 

A significant Chi-square was found in the area of written materials, a media category 
with which faculty indicated more familiarity. Faculty familiarity with materials in other 
media categories did not reach the .05 level of significance although an increase across 
most media categories was found. 

♦ 



TABLE 8 COMPARISON BY MEDIA TYPE OF FACULTY RESPONSES 
ABOUT FAMILIARITY WITH UNITS IN TEACHING AIDS 
PACKAGE, 1971-72 and 1972-73 



Media Category 


Percent Yes 
1971-72 (N = 87) 


Percent Yes 
1972-73 (N= 103) 


Percent 
Difference 


Written Material 
(8 Units) 


66 


67 


+ 1* 


Tape-Slide Programs 
(6 Units) 


79 


82 


+3 


Motion Pictures 
(5 Units) 


76 


85 


+9 


Audiotape Programs 
(3 Units) 


76 


75 


-1 



♦Significant Beyond the .05 Level 
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Availability of Units to Students 



Changes were found in the willingness of faculty to allow students to have available 
to them units in the Teaching Aids Package. The changes are shown m I able 9. 

In terms of faculty's making materials in the various media categories available for 
student use, only one area of significance was found as measured by the Chi-squarc 
statistic. Not surprisingly, this significant Chi-square was in the area of motion pictures, a 
modality which faculty indicated they were making more available for student usage. No 
statistical significance was found for the other media category modalities. However, shifts 
noted were in the direction of making all media categories more available for student 
usage. 

TABLE 9 COMPARISON BY MEDIA TYPE OF FACULTY RESPONSES 
ABOUT AVAILABILITY FOR STUDENT USAGE OF UNITS 
IN TEACHING AIDS PACKAGE. 1971-72 AND 1972-73 



Media Category 


Percent Yes 
1971-72 (N = 87) 


Percent Yes 
1972-73 (N= 103) 


Percent 
Difference 


Written Material 
(8 Units) 


40 


42 


+2 


Tape-Slide Programs 
(6 Units) 


59 


65 


+6 


Motion Pictures 
(5 Units) 


68 


76 


+8* 


Audiotape Programs 
(3 Units) 


46 


51 


+5 



*Significant Beyond the .05 Level 



Required Use of Units 

Changes were found regarding faculty reactions in requiring students to use units in 
the Teaching Aids Package. Changes are shown in Table 10. 

Three areas in media categories concerning the faculty's requiring students to use 
materials disseminated in the Teaching Aids Package showed change over the two-year 
study. Significant Chi-squares were reached in written materials, tape-slide programs, and 
motion picture categories for which the shifts were in the direction of not requiring 
students to use materials in the Teaching Aids Package. Although significance was not 
reached in the media category of audiotape programs, the direction was the same as for 
the three previously mentioned categories. 

Units Helping Students to Develop General 
Knowledge in Obstetrics and Gynecology 

Changes in faculty reactions to units in the Teaching Aids Package in helping 
students to develop general knowledge in obstetrics and gynecology were found. Changes 
are shown in Table 1 1 . 
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TABLE 10 COMPARISON BY MEDIA TYPE OF FACULTY RESPONSES 
ABOUT REQUIRING STUDENTS TO USE UNITS IN THE 
TEACHING AIDS PACKAGE, 1971-72 AND 1972-73 



Media Category 


Percent Yes 
1971-72 (N = 87) 


Percent Yes 
1972-73 (N= 103) 


Percent 
Difference 


Written Material 
(8 Units) 


13 


6 


_7* 


Tape-Slide Programs 
(6 Units) 


25 


16 


_9* 


Motion Pictures 
(5 Units) 


46 


40 


-6* 


Audiotape Programs 
(3 Units) 


16 


12 


-4 



♦Significant Beyond the .01 Level 



TABLE 1 1 COMPARISON BY MEDIA TYPE OF FACULTY RESPONSES 
ABOUT UNITS IN TEACHING AIDS PACKAGE HELPING 
DEVELOP GENERAL KNOWLEDGE OF OBSTETRICS AND 
GYNECOLOGY, 1971-72 AND 1972-73 



Media Category 


Percent Yes 
1971-72 (N = 87) 


Percent Yes 
1972-73 (N = 103) 


Percent 
Difference 


Written Material 
(8 Units) 


49 


40 


_9* 


Tape-Slide Programs 
(6 Units) 


68 


63 


-5 


Motion Pictures 
(5 Units) 


67 


74 


+7 


Audiotape Programs 
(3 Units) 


54 


47 


—7* 



*Significant Beyond .05 Level 



Chi-square studies were also completed to determine if shifts occurred during the two 
years regarding the materials distributed in the Teaching Aids Package as pertained to 
helping students to develop general knowledge. Significantly fewer faculty indicated that 
the media categories of written materials and audiotape programs helped students to 
develop general knowledge in obstetrics and gynecology. No significant Chi-squares were 
found in the areas of tape-slide and motion picture programs for the question posed. 
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Units Helping Students to Answer Questions of 
Residents and Staff 

Changes in faculty reactions to units in the Teaching Aids Package in helping 
students to answer questions of residents and staff were found. Changes are shown in 
Table 12. 

Chi-square studies were also completed to determine if shifts occurred during the two 
years regarding the materials distributed in the Teaching Aids Package as pertained to 
helping students to answer questions of residents and staff in the area of obstetrics and 
gynecology. Significantly fewer faculty indicated that the media categories of written 
materials, tape-slide programs, and audiotape programs helped students to answer 
questions of residents and staff. Although no significant Chi-squares were found in the 
areas of motion picture programs for the question posed, shifts were in the direction that 
faculty felt that motion picture units helped in this area. 



TABLE 1 2 COMPARISON BY MEDIA TYPE OF FACULTY RESPONSES 
ABOUT UNITS IN TEACHING AIDS PACKAGE HELPING 
ANSWER QUESTIONS OF RESIDENTS AND STAFF, 
1971-72 AND 1972-73 



Media Category 


Percent Yes 
1971-72 (N = 87) 


Percent Yes 
1972-73 (N = 103) 


Percent 
Difference 


Written Material 
(8 Units) 


42 


32 


-10** 


Tape-Slide Programs 
(6 Units) 


59 


51 


__g** 


Motion Pictures 
(5 Units) 


56 


63 


+7 


Audiotape Programs 
(3 Units) 


43 


39 


_4* 



♦Significant Beyond the .05 Level 
♦♦Significant Beyond the .01 Level 



Units Helping Students to Answer Questions on Formal Examinations 
in Obstetrics and Gynecology 

Changes in faculty reactions to units in the Teaching Aids Package in helping 
students to answer questions on formal examinations were found. Changes are shown in 
Table 13. 
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TABLE 13 COMPARISON BY MEDIA TYPE OF FACULTY RESPONSES 
ABOUT UNITS IN TEACHING AIDS PACKAGE HELPING 
ANSWER QUESTIONS ON A FORMAL EXAMINATION, 
1971-72 AND 1972-73 



Media Category 


Percent Yes 
1971-72 (N = 87) 


Percent Yes 
1972-73 (N = 103) 


Percent 
Difference 


Written Material 
(8 Units) 


38 


30 


-8* 


Tape-Slide Programs 
(6 Units) 


56 


48 


-8* 


Motion Pictures 
(5 Units) 


53 


58 


+5 


Audiotape Programs 
(3 Units) 


41 


36 


-5* 



*Significant Beyond the .01 Level 



Chi-square studies were also completed to determine if shifts occurred during the two 
years regarding the materials distributed in the Teaching Aids Package as pertained to 
helping students to answer questions on examinations in the area of obstetrics and 
gynecology. Significantly fewer faculty indicated that the media categories of written 
materials, tape-slide programs, and audiotape programs helped students to answer 
questions on examinations. Although no significant Chi-squares were found in the areas 
of motion picture programs for the question posed, shifts were in the direction that 
faculty felt that motion picture units helped in this area. 

Appropriateness of Unit Media Formats 

Changes in faculty reaction to the appropriateness of the media modality for the 
conveyance of content were found. Changes are shown in Table 14. 

In the area of the appropriateness of the media format for the content being 
conveyed, significant Chi-squares over the two-year period were found in the areas of 
written materials, tape-slide programs, and audiotape programs for which faculty 
indicated that the modalities were less appropriate. Again, faculty perceived motion 
pictures as more appropriate modalities for the conveyance of the content; however, 
statistical significance was not found. 

Overall Faculty Reactions Toward New Materials 

Over the two-year period, faculty were asked if they felt the need for a future 
Teaching Aids Package and if they would purchase a new package from departmental 
funds if the Steering Committee were to undertake the effort needed to construct one. 
Changes in faculty reaction are shown in Table 15. 
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TABLE 14 COMPARISON BY MEDIA TYPE OF FACULTY RESPONSES 
ABOUT APPROPRIATENESS OF THE FORMAT OF UNITS 
IN THE TEACHING AIDS PACK A OF FOR THE CON- 
VEYANCE OF THE CONTENT, 1971-72 AND 1972-73 



Media Category 


Percent Yes 
1971-72 (N = 87) 


Percent Yes 
1972-73 (N= 103) 


Percent 
Difference 


Written Material 
(8 Units) 


45 


40 


-5* 


Tape-Slide Programs 
(6 Units) 


63 


62 


-1* 


Motion Pictures 
(5 Units) 


64 


72 


+8 


Audiotape Programs 
(3 Units) 


43 


41 


-2* 



*Significant Beyond the .05 Level 



TABLE 1 5 COMPARISON OF FACULTY RESPONSES TO NEED FOR 

FUTURE TEACHING AIDS PACKAGES AND WILLINGNESS 
TO PURCHASE NEW MATERIALS, 1971-72 AND 1972-73 





Percent Yes 


Percent Yes 


Percent 


Area 


1971-72 (N = 87) 


1972-73 (N = 103) 


Difference 


Need for Future 








Teaching Aids 








Package 


85 


87 


+2 


Future Purchase of 








Teaching Aids 








Package 


74 


84 


+ 10 



No significant Chi-squares were found for the data presented above; however, the 
direction of change was favorable for constructing a new Teaching Aids Package. Most 
departments indicated they would purchase such materials from departmental funds. It 
should be noted, however, that many faculty who responded "yes" to purchasing new 
materials wrote comments that they would prefer to review units first and purchase only 
those items needed for men teaching programs. 

Overall Faculty Reaction to Teaching Aids Package 

Changes in faculty reactions to the Teaching Aids Package were found. Changes are 
shown in Table 1 6. 



32 



TABLE 1 6 COMPARISON OF FACULTY RESPONSES ABOUT OVERALL 
RATING OF THE TEACHING AIDS PACKAGE, 1971-72 
AND 1972-73 



Responses 


Percent 
1971-72 (N = 87) 


Percent 
1972-73 (N = 103) 


Percent 
Difference 


Excellent 


36 


26 


-10 


Good 


47 


53 


+6 


Fair 


12 


11 


-1 


Poor 


0 


1 


+ 1 


No Response 


6 


9 


+3 



No significant Chi-squares were found for data shown above. It does appear that 
materials were still viable in the second year; however, faculty tended to be somewhat 
more critical. 

Summary 

Faculty reactions to material sent in the Teaching Aids Package were assessed over a 
two-year period. Changes found to be significant at or beyond the .05 level as measured 
by the Chi-square statistic are listed below: 

1. Faculty indicated significant gain in familiarity with units in the written 
materials media category. 

2. Faculty indicated significant gain in making units in the motion picture category 
available for student use. 

3. Faculty indicated significant decrease in requiring students to use units in all 
media category areas except audiotape programs. 

4. Faculty indicated significant decrease as to the value of units in students' 
developing general knowledge of content, answering questions of residents and 
staff, and answering questions on formal examinations and appropriateness of 
media format for units in the written materials, tape-slide, and audiotape media 
categories. 
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CHAPTER 6 

STUDENT REACTIONS TO THE APGO TEACHING AIDS PACKAGE 



The Teaching Program Assessment form and the Faculty Assessment of the Teaching 
Aids Package form were sent to all medical school departments of obstetrics and 
gynecology participating in the project. Due to the large number of students in the 
clinical vears in obstetrics and gynecology (approximately 1 1 ,000), it was decided to 
gather information from only a small number of schools. It should be pointed out that 
the prime purpose for the dissemination of the Teaching Aids Package was not to provide 
students with a basic set of materials which could be implemented by all schools; rather, 
it was the intention of the Steering Committee to provide these materials in various 
modality formats in order to demonstrate to faculty alternative teaching/learning 
strategies. Even so, it was recognized that some faculty might make these materials 
available to students and that a representative group of students 1 reaction to these 
materials would be beneficial in determining possible future directions. It was pointed out 
earlier that, in nearly all teaching programs, the Steering Committee felt that faculty 
would first review materials and decide if the materials were appropriate for their 
teaching program. Accordingly, students probably would not have the opportunity of 
making a thorough review of all materials, as would faculty contact persons. Even with 
this limitation, it was determined to go ahead with this portion of the study. In order to 
maintain comparability of information, it was decided to attempt to make the Faculty 
Assessment of the Teaching Aids Package form and the Student Assessment of the 
Teaching Aids Package form as similar as possible, even though it was recognized that 
students would not have reviewed or had available to them many of the materials that 
faculty members had reviewed. 

Purpose, Methodology, and Sample 

The lack of information regarding student reactions to teaching/learning materials 
could seriously weaken the impact of any modality employed by a department. Medical 
students as a general rule, tend to come from the upper portions of their undergraduate 
classes and may be very discriminate as to their selection of learning strategies. Factors 
which may contribute to this idea include increased standards of living in the United 
States where children have grown up in an "electric" age of television, battery-operated 
radios, and tape recorders, and where public schools and colleges have utilized newer 
technologies in educational programs. With this in mind, the Steering Committee felt that 
student feedback about the Teaching Aids Package could have a beneficial effect as the 
Committee decided about possible further disseminations. As with the Faculty 
Assessment of Teaching Aids Package form, a sample of students for the academic years 
1971-72 and 1972-73 was obtained. Again, the plan would allow assessment of the 
materials to occur over time under various contingent conditions. All departments are not 
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alike, and it was anticipated that a wide sampling would be desirable in order to 
determine the efficacy of the Teaching Aids Package in a variety of learning 
environments. 

An attempt was made to obtain reactions from students from at least one 
department within each district of the Cooperative Teaching Network. In some cases, 
district chairmen decided to gather data from more than one school. However, a plotting 
of responses from departments that supplied student responses across the country 
matched closely with the actual dispersion of medical schools throughout the United 
States and Canada for both years when the forms were administered. 

During the 1971-72 academic year, Student Assessment of Teaching Aids Package 
forms were obtained from 164 students who were from 23 departments of obstetrics and 
gynecology in medical schools in the United States and Canada. It should be noted that 
some of the departments forwarded only a few forms from students and that the total 
clerkship for some of the departments concerned was not surveyed. During the 1972-73 
academic year, 12 schools surveyed students on their clinical clerkships and a total of 161 
students responded. Returns for the second year of the study were considered more 
reliable as the 12 schools reporting appeared to include all students on the clerkship 
where the forms were administered. In light of the above, the reader should be aware that 
the forms obtained did not represent a scientifically designed random sample of students. 
Rather, it represented more of a regional sample design. 

Chi-Square Comparison 

A Chi-square statistic program was used to determine if significant differences in the 
data reported in 1971-72 and 1972-73 existed. There were areas where significant 
statistical differences were noted. The Chi-square studies were limited to media categories 
in order to determine if modality preferences were indicated. However, changes in 
student reactions to each unit in the Teaching Aids Package are shown in Appendix C. 

The same procedures as used with faculty data were used to analyze information. 
Percentages were computed by multiplying the number of units in a media category by 
the number of usable forms returned, then dividing that value into the number of 
respondees who answered the item "y es >" and then multiplying by 100 to obtain a 
percent value. 



Student Use of Units 

Changes in student use of units in the Teaching Aids Package were found. Changes 
are shown in Table 17. 

In terms of students using materials in the Teaching Aids Package, two significant 
Chi-squares were found. These were in the areas of tape-slide and motion picture 
programs. In terms of the tape-slide programs, the direction was that fewer students were 
using those programs while the direction in terms of motion picture programs indicated 
that more students were using those units. Although significance was not obtained in the 
areas of written materials and audiotape programs, slight differences did occur. 
Essentially, student usage of these materials remained fairly constant over the two-year 
period. 
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TABLE 1 7 COMPARISON BY MEDIA TYPE OF STUDENT RESPONSES 
ABOUT USING UNITS IN TEACHING AIDS PACKAGE 
1971-72 AND 1972-73 



Media Category 


Percent Yes 
1 971-72 (N = 164) 


Percent Yes 
1972-73 (N = 161) 


Percent 
Difference 


Written Material 
(8 Units) 


28 


22 


-6 


Tape-Slide Programs 
(6 Units) 


54 


37 


1 7* 


Motion Pictures 
(5 Units) 


56 


54 




Audiotape Programs 
(3 Units) 


47 


34 


-13 



♦Significant Beyond the .01 Level 



Required Student Use of Units 

Changes in student responses to faculty's requiring them to use units in the Teaching 
Aids Package were found. Changes are shown in Table 18. 

The media categories of written materials, tape-slide programs, motion picture 
programs, and audiotape programs were all found to be significantly different using 
Chi-square statistics. The direction was that fewer students were being required to use 
media in all media categories. 



TABLE 1 8 COMPARISON BY MEDIA TYPE OF STUDENT RESPONSES 
ABOUT THEIR REQUIRED USAGE OF UNITS IN 
TEACHING AIDS PACKAGE, 1971-72 AND 1972-73 



Media Category 


Percent Yes 
1971-72 (N = 164) 


Percent Yes 
1972-73 (N = 161) 


Percent 
Difference 


Written Material 
(8 Units) 


8 


1 


-7* 


Tape-Slide Programs 
(6 Units) 


10 


3 


-7* 


Motion Pictures 
(5 Units) 


17 


7 


-10* 


Audiotape Programs 
(3 Units) 


1 1 


1 


-10* 



•Significant Beyond the .01 Level 
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Units Helping Students to Develop General Knowledge 

Changes were found in student reaction toward units in the Teaching Aids Package in 
helping them to gain general knowledge in obstetrics and gynecology. Changes are shown 
in Table 19. 

Student reactions over the two-year period to the Teaching Aids Package as it helped 
them to develop general knowledge in obstetrics and gynecology revealed no significant 
Chi-square changes. The direction was that fewer students in the 1972-73 academic year 
answered positively to this question. 

TABLE 1 9 COMPARISON BY MEDIA TYPE OF STUDENT RESPONSES 
ABOUT USEFULNESS OF UNITS IN TEACHING AIDS 
PACKAGE IN DEVELOPING GENERAL KNOWLEDGE 
IN OBSTETRICS AND GYNECOLOGY, 1971-72 AND 1972-73 



Media Category 


Percent Yes 
1971-72 (N = 164) 


Percent Yes 
1972-73 (N = 161) 


Percent 
Difference 


Written Material 
(8 Units) 


23 


19 


-4 


Tape-Slide Programs 
(6 Units) 


47 


32 


-15 


Motion Pictures 
(5 Units) 


53 


51 


-2 


Audiotape Programs 
(3 Units) 


43 


32 


-11 



Units Helping Students to Answer Questions of Residents and Staff 

Changes were found in student reaction toward units in the Teaching Aids Package in 
helping them to answer questions of residents and staff. Changes are shown in Table 20. 

Student reactions over the two-year period to the Teaching Aids Package as it helped 
them to answer questions of residents and staff in obstetrics and gynecology were 
analyzed using the Chi-square statistic. A significant Chi-square was found for the media 
category tape-slide programs. However, the direction was that fewer students in the 
1972-73 academic year answered positively to this question for other media categories 
also. 

Units Helping Students to Answer Questions on 
Formal Examinations 

Changes were found in student reaction toward units in the Teaching Aids Package in 
helping th^m to answer questions on formal examinations. Changes are shown in 
Table 21. 

Student reactions over the two-year period to the Teaching Aids Package as it helped 
them to answer questions on examinations in obstetrics and gynecology revealed little 
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TABLE 20 COMPARISON BY MEDIA TYPE OF STUDENT RESPONSES 
ABOUT UNITS IN TEACHING AIDS PACKAGE HELPING 
ANSWER QUESTIONS ON A FORMAL EXAMINATION, 
1971-72 AND 1972-73 



Media Category 


Percent Yes 
1971-72 (N = 164) 


Percent Yes 
1972-73 (N = 161) 


Percent 
Difference 


Written Material 
(8 Units) 


17 


14 


-3 


Tape-Slide Programs 
(6 Units) 


37 


26 


-11* 


Motion Pictures 
(5 Units) 


42 


41 


-1 


Audiotape Programs 
(3 Units) 


33 


24 


-9 



♦Significant Beyond the .01 Level 



TABLE 21 COMPARISON BY MEDIA TYPE OF STUDENT RESPONSES 
ABOUT UNITS IN TEACHING AIDS PACKAGE HELPING 
ANSWER QUESTIONS ON A FORMAL EXAMINATION, 
1971-72 AND 1972-73 





Percent Yes 


Percent Yes 


Percent 


Media Category 


1971-72 (N = 164) 


1972-73 (N = 161) 


Difference 


Written Material 








(8 Units) 


15 


13 


-2 


Tape-Slide Programs 








(6 Units) 


32 


22 


10 


Motion Pictures 








(5 Units) 


38 


36 


-2 


Audiotape Programs 








(3 Units) 


26 


20 


-6 



changes, and no significant Chi-squares were found for the media categories. Yet, the 
direction was that fewer students in the 1972-73 academic year answered positively to 
this question. 



Student Reactions to the Appropriateness of the Media 
for the Conveyance of the Content 

Changes were found in student reaction to the appropriateness of the media for the 
conveyance of the content. Changes are shown in Table 22. 
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TABLE 22 COMPARISON BY MEDIA TYPE OF STUDENT RESPONSES 
ABOUT APPROPRIATENESS OF THE FORMAT OF UNITS 
IN THE TEACHING AIDS PACKAGE FOR THE CON- 
VEYANCE OF THE CONTENT, 1971-72 AND 1972-73 



Media Category 


Percent Yes 
1971-72 (N= 164) 


Percent Yes 
1972-73 (N = 161) 


Percent 
Difference 


Written Material 
(8 Units) 


22 


18 


-4 


Tape-Slide Programs 
(6 Units) 


43 


29 


-14 


Motion Pictures 
(5 Units) 


48 


48 


0 


Audiotape Programs 
(3 Units) 


36 


28 


-8 



No significant Chi-squares were found for the two-year study period in terms of the 
appropriateness of the format for the conveyance of the content according to media 
categories. However, the direction was that fewer students in the 1972-73 academic year 
answered positively to this question for all media categories except that of motion 
pictures for which no change was found. 



Overall Student Reaction to Teaching Aids Package 

Changes in student reaction to the Teaching Aids Package and the need for a new 
Teaching Aids Package were found. Changes are shown in Tables 23 and 24. 

It was of interest to note that student reactions toward the quality of the Teaching 
Aids Package and to the need for a future Teaching Aids Package did not change 
significantly over the two-year period. Therefore, a large majority of the students felt that 
the Teaching Aids Package was either excellent or good and that a future Teaching Aids 
Package was needed. 

Summary 

Student reactions to materials sent in the Teaching Aids Package were assessed over a 
two-year period. Changes found to be significant at or beyond the .05 level as measured 
by the Chi-square statistic are listed below: 

1 . Students indicated significant decreases in the use of units in the tape-slide and 
audiotape media categories. 

2. Students indicated significant decreases in their being required to use units in all 
media category areas. 

3. Students indicated significant decrease in units in the tape-slide media category 
helping them to answer questions of residents and staff. 
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TABLE 23 COMPARISON OF STUDENT RESPONSES ABOUT 
OVERALL RATING OF THE TEACHING AIDS 
PACKAGE, 1971-72 AND 1972-73 





Percent 


Percent 


Percent 




1971-72 (N = 164* 


17/ L- 15 1 IN — 161) 


Difference 


Excellent 


29 


31 


+2 


Good 


48 


53 


+5 


Fair 


15 


8 


-7 


Poor 


2 


1 


-1 


No Response 


6 


7 


+ 1 



TABLE 24 COMPARISON OF STUDENT RESPONSES ABOUT NEED 
FOR FUTURE TEACHING AIDS PACKAGES 



Responses 


Percent 
1971-72 (N = 164) 


Percent 
1972-73 (N = 161) 


Percent 
Difference 


Yes 


75 


77 


+2 


No 


16 


17 


+ 1 


No Response 


9 


6 


-3 



41 



CHAPTER 7 

INTRAORGANIZATIONAL COOPERATION AND 
THE THRUST IN NEW DIRECTIONS 

The original objectives of the Ad Hoc Committee, later to become known as the 
Steering Committee for Cooperative Teaching of the APGO, were expanded over the 
three-year period of its existence. The main objective was to include all departments of 
obstetrics and gynecology in medical colleges in the United States and Canada in their 
efforts. The activities of this group have also increased, especially in relation to other 
organizations serving undergraduate medical education in obstetrics and gynecology, 
groups representing other disciplines in medical education, and national groups 
representing all disciplines commonly taught in medical schools and health science 
centers. It seems appropriate to describe the strategies for obtaining intraorganizational 
cooperation and input and to indicate new directions being pursued by the Steering 
Committee. 

During its first year of operation, the period between July 1969 and July l c )70, 
intensive efforts were made by the membership to develop and select materials. As the Ad 
Hoc Committee worked with the Association of Professors of Gynecology and Obstetrics 
and as increased interest was expressed by other medical school departments of obstetrics 
and gynecology in participating in the activities of the Steering Committee, attention was 
focused on dissemination of a "Sampler Package" and assessing its impact. These later 
activities were the focus of the second two years of the group's energy. However, once 
the Steering Committee had implemented its dissemination and assessment effort, new 
tasks and thrusts became more critical. 



Intraorganizational Cooperation 

In July of 1971 , Stead and others published their study on "Educational Technology 
for Medicine: Roles for the Lister Hill Center. "''(34) This comprehensive study added 
direction to the efforts of the Steering Committee. The stimulus for attempting to create 
a United States and Canadian network involving all departments of obstetrics and 
gynecology in all medical schools was certainly influenced by that report. A second 
report entitled "Educational Technology for Medicine: Academic Institutions and 
Program Management,'Y72 / / addressed itself to many of the problems faced by users of 
newer technology for educational purposes. These documents, while addressing a much 
broader audience, raised relevant issues for the Steering Committee. It should be noted 
that a Steering Committee member was on the committee that worked on the first 
document referred to above and that advance copies of the second document were 
provided to the Steering Committee and certainly influenced directions being pursued. 
The staff of the Association of American Medical Colleges worked with the Steering 
Committee in a framework of mutual sharing of information and ideas. In such a manner, 
the efforts of the Steering Committee and participants on the network were integrated 
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into an interdisciplinary effort in medical education. 

While it is important to maintain communication lines with other disciplines and 
organizations which represent all of the disciplines involved in medical education efforts, 
the present structure of medical education requires even greater efforts to coordinate 
activities within a specified discipline. The Steering Committee has been most fortunate 
in avoiding serious internal problems within the discipline throughout its existence. 
Several factors contributed to its ability to avoid potential conflicts, and the strategy tor 
accomplishing this aim probably needs to be stated. 

When the Ad Hoc Committee was founded, members of that committee belonged to 
a variety of national organizations - all of which had some interest in the education of 
students and practitioners within the discipline. As the Ad Hoc Committee worked at its 
original objectives, key persons and other organizations were kept informed as to its 
activities through the Newsletter communication technique. Eventual affiliation with the 
Association of Professors of Gynecology and Obstetrics was appropriate because persons 
from other disciplinary organizations held membership in that organization. For example, 
members of the Council on Residency Education in Obstetrics and Gynecology (CREOG) 
are associated with the Association of Professors of Gynecology and Obstetrics and are 
represented in the Steering Committee membership; a member of the Audiovisual 
Committee of the American College of Obstetrics and Gynecology (ACOG) is a member 
of the Association of Professors of Gynecology and Obstetrics and of the Steering 
Committee; several members of the Steering Committee conduct examinations for the 
American Board of Obstetrics and Gynecology, and two members consult with the 
National Board of Medical Examiners (NBME); one member has been very closely 
associated with the American Association of Obstetricians and Gynecologists Foundation, 
Inc. (AAOG), Kennedy Foundation Curriculum Committee efforts and has served to 
insure liaison between the efforts of that group and the Steering Committee; and finally, 
a leading producer in the Southern Medical Schools Consortium, "Self-Instructional 
Materials Project," recently was invited to join the Steering Committee in order to add 
the experience of that vigorous group. These important lines of communication to other 
groups markedly facilitated communication, provided important input, and catalyzed the 
development of ideas, programs, and a sense of accountability. 

Even within the Steering Committee, disagreement exists over specific policies. To 
mediate, as much as possible, potential damaging effects of such honest differences of 
opinions, one strategy developed by the Steering Committee membership has worked 
rather well. Basically, it is the concept of centralized decentralization. That is to say, each 
Steering Committee member, by virtue of his supervision of a number of schools within a 
geographical district, has been able to determine the evaluative and productive efforts of 
participants of the network within his district. This has led to a sense of responsibility on 
the part of each Steering Committee member as to what the proper functions of the 
central project staff ought to be. resulting in centralized functions being more "service 
oriented" rather than being "directed oriented" in nature. There is a value system 
expressed above which is firmly held by the Steering Committee, that decentralized 
power bases lead to more effective operation for centralized service activities. The Project 
Center, then, has been designed to serve in a "helping relationship" to the Steering 
Committee members and the districts they represent. It is also noted that each district 
chairman serves not only the Steering Committee for Cooperative Teaching of the APGO, 
but other organizations which have made seminal contributions to medical education 
within the discipline. Rather than compete with activities of other organizations, it has 
been a Steering Committee policy to encourage and integrate these activities to serve the 
network participants. While honest differences of opinion exist and, in all likelihood, will 
continue to exist, the Steering Committee shall work to bring reality to the concept of 
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cooperative efforts. In the final analysis, the benefits accrue to the medical student, the 
medical practitioner in the area of obstetrics and gynecology, and ultimately to the 
patient. 



Thrusts in New Directions 

By November 1972, it was apparent to the Steering Committee that its initial 
objectives were being met and that the assessment procedures and data obtained indicated 
that the network was, in fact, established and functioning. It was also recognized that the 
Steering Committee needed to change its emphasis from attention to dissemination and 
attitudinal assessment back toward its original aims of production, selection, 
communication, and evaluation in developing a new Teaching Aids Package which could 
be experimentally tested. By February 1973, efforts in these old, yet new directions were 
begun. 

The Steering Committee devised a media evaluation form for evaluating products 
which were becoming more and more available. These evaluation forms are being 
collected and copies of them are shared with the Audiovisual Committee of the American 
College of Obstetrics and Gynecology. By April of 1974, tabulations of these media 
evaluation forms will be provided to the Office of Audiovisual Educational Development, 
Bureau of Health Resources Development, and the National Medical Audiovisual Center 
of the National Library of Medicine to support them in their major clearinghouse efforts. 

The Steering Committee has selected twelve priority objectives from the Guide to a 
Basic Curriculum in Obstetrics and Gynecology(20) for which it hopes to identify and/or 
produce educational materials. In addition, the Committee hopes to develop formative 
tests related to each item selected and a final summative evaluation instrument to test the 
learning effectiveness of a new Teaching Aids Package as an integrative instructional 
system. The twelve terminal objectives selected from the forty terminal objectives 
published in the document referred to above are listed below. 

1 . Given an adult female patient, the student will obtain a complete 
history, including the following: (a) pelvic pain, bleeding, discharge; 

(b) menstrual history; (c) obstetrical history; (d) contraceptive 
history; and (e) sexual history. The student will have demonstrated 
competency in obtaining a complete history if he records information 
on all items as described in the enabling objectives. 

2. Given a female patient, the student will perform a complete pelvic 
examination and will obtain a cytologic smear of the cervix. 

3. Given a patient who has missed two menstrual periods, by means of 
history and physical findings, the student will be able to diagnose 
pregnancy. 

4. When presented with an uncomplicated obstetrical patient, the 
student will be able to inform her about: (a) diet; (b) hygiene; 

(c) exercise; (d) preparation for labor; (e) family planning; (f) danger 
signs of pregnancy ; and (g) use of drugs. 

5. Given a patient in labor, the student will be able to monitor the 
patient until delivery and differentiate between normal and abnormal 
labor. 

6. Given a patient in the postpartum clinic, the student will be able to 
reassess her at the end of the puerperium by: (a) performing a 
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physical examination; (b) performing a cytologic smear of the cervix: 
(d) ordering a hematocrit; and (d) counseling with regard to family 
planning. 

7. Given a patient with vaginal bleeding in the first trimester of 
pregnancy, the student will diagnose and differentiate between 
abortion and ectopic pregnancy. He will then write out the 
appropriate plan of management. 

8. Given a patient in third trimester of pregnancy, the student should be 
able to recognize pre-eclampsia and outline a plan of management. 

9. Given a case, the student will be able to make the diagnosis and 
describe the treatment of vulvovaginitis. 

1 0. Given a patient with symptoms of menopause, the student will be able 
to: (a) elicit an appropriate medical history; (b) perform an 
appropriate physical examination; (c) order appropriate laboratory 
tests; (d) interpret these tests: and (e) make a differential diagnosis. 

11. Given a patient with dysmenorrhea or the premenstrual tension 
syndrome, the student will be able to identify the symptoms, 
ascertain whether there is an organic cause, and plan appropriate 
management. 

12. Given actual or illustrative cases, the student will be able to decide 
whether a contraceptive history should be taken and actually take 
one, decide whether to prescribe a contraceptive method, decide 
which contraceptive method is most appropriate for particular 
patients in illustrative cases, and discuss with patients the advantages, 
disadvantages, and potential contraindications of contraceptive 
methods. 

Individual Steering Committee members have selected objectives and are working 
independently and in conjunction with one another to identify and/or produce media for 
them. While it may not be possible for materials to be developed for all objectives, it is 
hoped that a future Teaching Aids Package will cover a minimum of six of the twelve 
priority objectives as far as didactic information is concerned. 

Several groups have been making progress in these areas and some materials are 
already under development. Materials in the original Teaching Aids Package have been 
reviewed in terms of their applicability to these new objectives, and the correlation was 
published in the Cooperative Teaching Bulletin, Volume 3, Number 2, May 1973 issue. In 
addition, previous review efforts by subcommittees of the Steering Committee identified 
some excellent materials that will undoubtedly become a part of the new Teaching Aids 
Package. A great deal of assistance in accomplishing these tasks has been obtained from 
the Office of Audiovisual Educational Development, Bureau of Health Resources 
Development, and the National Medical Audiovisual Center, National Library of 
Medicine. Professional educators from those offices are in the process of reviewing efforts 
of the Steering Committee in developing materials and are providing critiques as to plans 
and prototype versions of materials being produced. In addition, Steering Committee 
district chairmen often rotate the location of their workshops throughout their district, 
and experts from learning resource centers and divisions of research in medical education 
have been, and shall be, very helpful in providing the technical and educational expertise 
needed to produce quality materials. 

It is planned that the new Teaching Aids Package will be completed and forwarded to 
15 test institutions by April 1974. Resultant data, along with faculty and student 
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reaction instruments, should provide information as to the viability of the new materials 
under a controlled setting which did not exist prior to the distribution of the original 
Teaching Aids Package. A strategy for further dissemination of the new Teaching Aids 
Package will probably not be resolved until the efficacy of the new materials is 
determined. 

The Steering Committee is also engaged in two other supportive projects for medical 
education. It is developing for the American College of Obstetrics and Gynecology a 
package of materials for learning about human sexuality, contraception, and famil) 
planning, and it is anticipated that distribution of this package to all medical schools on 
the Cooperative Teaching Network will occur prior to the end of 1973. Included with the 
package will be a post-test to measure knowledge in this area. In addition, the American 
College of Obstetrics and Gynecology will use these materials in workshops especially 
designed for physicians other than Board-Certified obstetricians and gynecologists, 
thereby ensuring that materials in this vital area are readily available to practicing 
physicians who offer family practice or general medical care to patients. 

The Steering Committee is also working with Ortho Pharmaceuticals to revise 
materials prepared for nurse practitioners in the area of obstetrics and gynecology, in 
order that they may be used with medical students. An earlier review of some of these 
materials by the Steering Committee led to the conclusion that these materials, slightly 
modified, could be of value to medical students. A prime reason that the Steering 
Committee reached such a decision was the fact that the development of materials by 
Ortho Pharmaceuticals for use with nurse practitioners was based upon a task analysis of 
functions required to deliver health care in the area of obstetrics and gynecology to 
patients. While the Steering Committee is not prepared to endorse the idea that some of 
the functions presently performed by physicians should rest with a less trained individual, 
they are prepared to work with these materials in order that the medical student can 
profit from the obviously well-prepared and tested set of learning materials. 

It is apparent from what we have discussed above that the Cooperative Teaching 
Network in Obstetrics and Gynecology has much to do. The Steering Committee can > 
contribute; however, the network's eventual course will be decided by the faculty who 
contribute their time to improving undergraduate education in obstetrics and gynecology, 
and a continuation of intergroup cooperation. In the final analysis, these two factors are 
both the great strengths and the possible hazards of a networking approach to cooperative 
sharing of educational materials. 
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CHAPTER 8 
CONCLUSIONS 



It has been four years since the inception of the idea of cooperative teaching in 
obstetrics and gynecology by the Steering Committee in July 1969. 

A basic assumption of the Steering Committee is that medical student teaching is the 
primary mission of medical college departments. Correlated and integrated interests such 
as patient care, community service, and research are critical and necessary but must play a 
secondary role to the teaching mission. 

The Steering Committee for Cooperative Teaching, Association of Professors of 
Gynecology and Obstetrics, is working vigorously with its colleagues and other 
professional groups both in medicine and other behavioral disciplines. Conclusions 
reached from the four years of experience are: 

1. Establishment and Implications of a Biomedical Communication Network. The 
most significant accomplishment of the Steering Committee is the establishment of the 
first comprehensive biomedical communication network for educational purposes. All 
departments of obstetrics and gynecology in all medical colleges in Canada and the 
United States are members of the network, receive a quarterly Bulletin for Cooperative 
Teaching, and have received a set of learning materials because of the efforts of the 
Steering Committee. The districting plan, in which each Committee member has 
responsibility for coordinating efforts for departments in his geographical area, adds 
structure and decentralization to a unified effort of the Association of Professors of 
Gynecology and Obstetrics. 

This accomplishment required cooperation of Steering Committee members; Council 
members of the Association of Professors of Gynecology and Obstetrics; professors from 
each department; the Fund for Medical Education, which provided additional 
nongovernmental resources to make the concept of networking a reality; and support 
(both financial and advisory) from the Office of Audiovisual Educational Development of 
the Bureau of Health Resources Development, and the Lister Hill National Center for 
Biomedical Communications and the National Medical Audiovisual Center of the National 
Library of Medicine, which identified the need for providing support to the networking 
effort of the Association of Professors of Gynecology and Obstetrics. 

Participating departments had new and imaginative pedagogical techniques. Our 
study showed that media sent to nearly all departments have been well received and that 
student data from a sample of 27 departments indicate media have been used by students, 
although availability of any item was a decision made by local faculty based on local 
needs. Evidently, faculty felt that student learning could be facilitated using educational 
media. 

Workshop sessions and early publications by Steering Committee members 
emphasized the critical role of faculty in not allowing educational media to substitute for 
student-faculty contact. Rather, they emphasized the use of educational media to 
enhance instruction and encourage students and faculty to reach for higher levels of 
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intellectual and skillful application of the arts and science of medicine. The newer 
technology in medical education should embellish, not replace, faculty-student contact. It 
seems reasonable to assume that the faculty have been relieved of repetitive tasks and 
have spent the increased time as shown in our data in meeting individual student needs, 
challenging the advanced, and helping the laggard. 

Participants at workshops freely communicated problems and possible solutions to 
improve teaching programs. They formed relationship which continued after workshops 
in the preparation of new and better educational materials. The benefits of these 
production efforts are potentially of great significance. Excellent materials produced by 
professors working cooperatively are readily available and quickly communicated to all 
the participants of the network. While the materials being produced and shared may not 
be of the highest quality, they are in all likelihood improvements over materials sent in 
the original Teaching Aids Package and materials sent in other distributions by other 
groups. The Steering Committee recognized that much work remains to be done in order 
to produce and disseminate truly excellent educational materials, but the future promises 
closer relationships between professors through communicating at future workshops, 
continuing dialogue using the Bulletin for Cooperative Teaching, joint projects between 
schools and consortia, and new, unified, tested Teaching Aids Packages for objectives 
found in the Guide to a Basic Curriculum in Obstetrics and Gynecology. (20) 

Recognizing these facts and hopes, the Steering Committee believes the student must 
have available to him alternative pathways to learning. Assessment procedures need only 
indicate that some, not all, students find a particular program very helpful in mastering an 
educational objective. It is the objective that should be the focus, not the method of 
learning. The medium is not as important as the content of the medium. Our experience 
with the Teaching Aids Package supports this view. 

Both faculty and students on the one hand expressed a need for more teaching aids 
and on the other hand commented that the format was inappropriate. This is not 
surprising, but refreshing! The packages were selected to demonstrate technology; they 
were first-generation attempts; they served the purpose of introducing to students and 
faculty alternate learning pathways. In the future, the emphasis will be on the objectives, 
the content, and the appropriateness of the format. Close attention must be paid to the 
transfer of training, i.e., to the application of knowledge in the real world. To ensure this, 
we will need valid evaluation instruments. Heretofore, resident and staff questions and 
formal examination questions were not directed to and were not measuring the same 
ability, knowledge, and skill used in actual practice. Educational programs and assessment 
programs of health care delivery systems need to be coordinated. 

2. Problems in Need of Solution. Although the Cooperative Teaching Network has 
filled a need to enhance teaching-learning programs in obstetrics and gynecology, the 
filling of such a need has created new problems which must be faced and resolved. Four 
major areas appear to be most pressing at this time. They are (1) providing for future 
dissemination of teaching packages; (2) continuation of Steering Committee production 
activities; (3) meeting the needs of developing institutions - organizing teaching-learning 
programs; and (4) ultimate evaluation procedures. 

3. Providing for Future Materials. The first Teaching Aids Package disseminated by 
the Steering Committee was fully paid for by the United States Government. At issue and 
yet to be resolved is the question, "Who will pay for future disseminations?" While it is 
possible for many departments to purchase materials, most, because of the extreme 
strains on medical school budgets, are faced with serious financial cutbacks. In addition, 
there are some departments which have made little use of the materials from the 
dissemination, and one may hypothesize that these departments are in the most dire need 
of such materials. Potential supporters for disseminations to departments on the network 
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are the United States Government, the Canadian Federal Government, private 
foundations, professional organizations, contributions from business and industry, a 
pooling of resources from all departments on the network, or combinations of resources. 
It is not within the wisdom of the Steering Committee to recommend "who should foot 
the bill," even though users are sharing and must continue* to share the cost. However, the 
problem needs to be resolved in order to continue and accelerate progress in improving 
teaching-learning programs in obstetrics and gynecology. In order to accomplish this, 
mechanisms must be developed by winch talented, imaginative, and competent 
professionals (i.e., subject-matter and pedagogical experts) can combine to produce, test, 
and refine educational materials so that all students receive the benefits of the best that is 
available. 

4. From Assessment to Evaluation. Early efforts of producers of educational media, 
working individually, toyed with new ideas and techniques in poorly controlled 
situations. While materials were shared between producers as early as 1969 on an 
organized basis, little attempt was made to develop an evaluation procedure utilizing 
experimental and control groups in obstetrics and gynecology. Because of the efforts of 
the last four years, such experimentation is now feasible. While attitudes and reactions of 
both faculty and students to materials will continue to be valid measurement parameters, 
the evidence of changes in student performance based on methodology of learning must 
become an integral part of the development of educational materials in obstetrics and 
gynecology. It is to this end that Steering Committee members will focus their efforts for 
the next few years. 

Another major objective for the Steering Committee evolved. The Steering 
Committee will continue to encourage cooperation between all organizations and 
consortia interested in the undergraduate curriculum in obstetrics and gynecology. The 
excellent sessions conducted by the Kennedy Foundation Curriculum Committee and the 
American Association of Obstetricians and Gynecologists. Inc., in teaching professors 
how to teach, and the efforts by the Southern Medical Schools Consortium 
"Self-Instructional Materials Project" in teaching professors to write objectives, write 
tests for objectives, and write materials which teach the objectives and are tested by the 
tests, have also contributed to the expertise now evident in departments of obstetrics and 
gynecology in the United States and Canada. 

5. The Elusive, Ultimate Criteria. The final major problem concerns assessment and 
evaluation of instructional programs. Criteria for the "excellent physician" are not clearly 
delineated and may never be. However, means of relating teaching programs to improved 
patient care based on physician intervention are greatly needed. This is the greatest 
question. Its possible solution rests in the realm of longitudinal educational research. 

Only by continuing to have a viable group of content and educational experts 
working cooperatively over time can such solutions be found. It is the intention of the 
Steering Committee to try to resolve some of these problems and to make the long-term 
effort required of it. 
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